Convert Experiences: Helping Organizations of All Sizes Run More Tests Across Diverse Channels

Our crew never rests—we’ve been cooking up tons of new features to help you leapfrog ahead with your experimentation programs. After navigating 2023’s industry shakeups (Google Optimize sunset, anyone?) and major growth, we didn’t rest on our laurels—in 2024 we’ve launched even more product updates to equip you with peak testing technology.

Convert Says: Experimentation Is Human Nature™

We say this because experimentation is a cycle that is ubiquitous in our daily lives.

We observe a set of events, suspect a causal link, hypothesize why this link exists, and then take action. This cycle of observation, hypothesis, and action is present in everything we do, from switching to a ‘better’ workout routine or tweaking a cooking recipe. It’s like climbing a mountain of wins.

All human progress can be traced back to curiosity and experimentation. We are curious about meaning and purpose. This compels us to understand why things happen the way they do.

That’s why we see the same in businesses run by humans—experimentation in marketing, sales, product development, customer experience, pricing, etc. (Not like I’ve seen any business run by aliens but I bet they would do the same).

Hence, any platform or tool that wants to support proper experimentation should:

Free up resources (funds) for people and processes

Your testing tool is just one part of your experimentation program. And it is not even the most important part.

As Steph Le Prevost points out below (and many other experts concur), people and processes are the bedrock of validation and causal relationship-seeking culture.

In our “How to Optimize Each Stage of Your Experimentation Journey” event, Jeremy Epperson also argued that tool choice should not be the first consideration. Instead, the focus should be on the skills, process, and team building of how we actually execute against experimentation.

Allow for testing across all growth channels and motions

Here’s a bit of context to grasp what this means:

Elena Verna emphasizes the importance of leveraging all growth motions and levers in your growth model. And experimentation plays a role in all three growth motions.

• Product-led growth: To identify which product features drive user engagement and retention
• Marketing-led growth: To find the most effective marketing strategies or campaigns that drive user acquisition and conversion
• Sales-led growth: To find the most effective sales tactics/strategies — such as what pricing model increases the average deal size or close rate

Experimentation is, after all, an amplifier — improving what’s already working, spotting ideas that aren’t worth rolling out, and unearthing innovation to revamp what’s effective but gradually regressing to the mean.

All of which are elements driven by a comprehensive experimentation approach that permeates every aspect of your business and product. This is also known as full stack experimentation.

If your experimentation tool is built with your business’s best interest at its core, it should be able to support testing across all these growth motions.

Shorten the time from observation to insight to knowledge

This may manifest in exploration tools like heatmaps that don’t just show “what’s happening”, but also connect the “why it’s happening” to actionable next steps (i.e., ideas to test).

It also involves giving you the power to unlock data from any touchpoint along the customer’s journey, even if it is a 3rd party platform.

Simplify test deployment

This can be achieved by offering hyper fast response times, so you and your team aren’t left twiddling your thumbs through inordinately long waits and then subjected to frustrating scripted fluff responses.

Instead, your tool should increase time to action/learning; or slash the time you take to launch a test and get back results — an important program efficiency metric.

It should stay one step ahead of privacy issues which add a layer of hesitation to running tests because of either skewed results, steep fines, or heavy criticism (like in the case of Facebook’s infamous 2012 experiment).

At Convert, we’ve worked extensively on this aspect, as detailed in our blog post on respecting visitor privacy during experimentation.

All the Recent Additions to Convert Experiences (And Why They Matter)

Now, let’s get into our new additions and why you should be excited about them. Starting with the most recent:

Sequential Testing

Do you want to use Frequentist stats, but fixed horizon tests aren’t agile enough for the fast decisions you want to make? We’ve got you covered.

One- and two-tailed tests are no longer your only choices when you go to Stats & Settings in the Experiment Summary window in Convert.

We’ve added Sequential testing!

With this set, you can evaluate the results of your A/B tests at any point after reaching a minimum number of visitors.

This minimum number of visitors, a tuning parameter you can edit, ensures you can make faster decisions without risking premature conclusions. It still maintains statistical rigor with a stringent error rate control, so you can trust the findings from your tests.

This is huge because, with sequential testing, you can deploy winning variants faster or shut down variants that are unlikely to outperform the control. That earns more revenue for you as you deploy a winner in one case and stop burning money in the other.

AI Wizard

This was our first dive into the world of Generative AI. AI Wizard is a widget that sits in Convert’s Visual Editor as your partner for generating copy alternatives with speed.

Convert’s AI Wizard in the Beta Visual Editor

In the Visual Editor, click on an element on your page, such as your website headline, and you’ll find the AI Wizard tab in the left sidebar. Just click on it.

The AI Wizard helps you rewrite headlines and paragraphs of text using various models: the Seven Persuasion Concepts, the Fogg Behavior Model, and the most popular copywriting formats. This means you can swiftly craft alternate versions of your copy for A/B testing.

But let’s be clear: The AI Wizard is not here to replace strategic research, theme identification, or a seasoned copywriter’s strategic deliberation over word choice. Rather, it’s your companion for surfacing different angles to explore your existing web copy, all based on proven copywriting, persuasion, and behavior models.

And it is available to all Convert plans purchased in 2023 and beyond.

Full Stack Experimentation

Full stack and server-side testing are here to stay.

Because server-side testing can get around multiple cookie and tracking issues. And full stack equips organizations with the ability to not only test UI changes but also permeate the willingness to let data guide decisions when it comes to investing in features and even in determining the business model adopted.

We dive into all the details in our full stack experimentation guide but here’s the rundown of why server-side and full-stack are so important. They:

• Fuel continuous learning and delivery of higher quality products,
• Provide a comprehensive view of the customer experience across channels, devices, and interactions,
• Help manage the risk of focusing solely on a single success metric,
• Accelerate innovation, reduce time to value, and minimize operational and performance risks,
• Enable deeper testing of website aspects such as logic and functionality,
• Allow for product feature testing, enabling the testing and refinement of new ideas and features,
• Offer rich aggregation of business-wide insights for data-backed decision-making,
• Allow testing of components coming from the backend of websites,
• Allow testing of certain aspects of mobile apps, and more.

Companies like Etsy, Microsoft, Uber, Amazon, Facebook, LinkedIn, Google, Twitter, Netflix have used full-stack experimentation to fuel innovation and improve the customer experience on their sites. They understand that moving away from front-end changes only can open up a sea of strategic possibilities — allowing development to more directly support and inform the evolution of the business. We understand that too. And if you don’t already, you should too.

That’s why we’ve decided to take things to a whole new level with our Full Stack features—with server side experimentation, feature flags, and rollouts. Convert is truly a comprehensive experimentation platform that allows you to test and optimize across your entire tech stack, growth channels and motions.

You can now roll out new features to a subset of users, test their impact, and then decide whether to roll them out to everyone or not. You get to see how your users react and then decide whether to go for a full release.

Our SDKs allow you to integrate Convert’s robust features into your own applications. We currently support JavaScript (most popular JS frameworks are supported) and PHP SDKs.

Customizable Bayesian & Frequentist Stats Engines

You can now tailor your stats engine to your preferences.

For instance, for Frequentist, you can choose one-, two-tailed, and sequential test types, along with comparison corrections like Bonferroni and Sidak and your desired confidence level.

Alternatively, opt for Bayesian reasoning to focus on the Chance to Win instead of the more traditional Confidence value.

Convert’s Customizable Bayesian and Frequentist Stat Engines

You now have the power to pop the trunk and mod the engine running your experiments — putting even more control in your hands.

Now, you may be wondering: When do you use Frequentist and when do you go with Bayesian? Contrary to popular opinion, it isn’t about the sample size. It’s about your comfort with uncertainty and your need for actionable results.

Bayesian is great when you’re comfortable with a degree of uncertainty and need to make decisions quickly. Frequentist is your go-to when you need more controlled results and are willing to do proper test sample size prepping and resist the temptation to peak.

Frequentist is the sturdy oak tree, while Bayesian is like the flexible bamboo.

With the power to choose the stats engine you want for the experiment you’re running, Convert is supporting you by shortening the time from observation to insight to decision-making.

Visual Editor Updates

Now, each individual change is connected on a DOM level, which means less duplication, easier re-editing, and deletion of specific past changes without having to edit the entire code.

This is a game-changer for those who want to make quick and precise changes to their A/B testing experiments. You can fine-tune the elements of your website with surgical precision.

This further simplifies test deployment for CRO teams. You spend less time and effort in launching and modifying tests, and free up time for action and learning.

Locations

Site Area, your familiar tool for specifying the exact places where your experiments trigger, is now Locations.

Here’s what this means:

• Old experiences with “Site Area” will continue to run without changes.
• New experiences will have “Locations” (instead of Site Area).
• Locations are reusable. They can be saved and used across experiences to save time.
• Site Area has been phased out.

We hope this helps you build reusable and more structured CRO programs for your team and clients.

And in the spirit of freeing up resources for people and processes, we’re thrilled this means you get to save time you would otherwise have spent building new locations for every experience.

API V2

The API V2 is a major upgrade that allows you to easily access account information as well as reporting information, while giving you access to the full experimentation lifecycle.

Everything that can be done through our UI can also be done via the API.

You can

• create an experiment programmatically,
• run custom analysis on experiment results,
• build customized integrations and workflows,
• create feature flags from an automated script,
• build custom dashboards of feature test results,
• connect your experiments to other project management tools, and more.

You can create projects, or completely automate the whole experiment pipeline without ever touching the Convert UI, up to you.

Native GA4 Integration

We’re not just keeping up with the times, we’re leading the charge with our native GA4 integration. This isn’t just a minor update, folks.

With this integration, you can seamlessly connect your Convert Experiences with GA4, making it easier than ever to track and analyze your experiments.

No more juggling between platforms or wrestling with incompatible data. Everything you need is now in one place, working together in perfect harmony to shorten your time from observation to insight to knowledge.

But what does this mean for you? You can now leverage the power of GA4’s advanced analytics capabilities right from your Convert Experiences dashboard.

You can track user interactions, analyze user behavior, and gain deeper insights into your experiments, all without leaving Convert Experiences.

And the best part? This integration is based on Google’s official integration API documents, which means it’s secure, reliable, and designed to work seamlessly with GA4.

Plus, Convert is one of the first A/B testing providers to have an official functional integration with GA4. So, you’re not just getting a cutting-edge feature, you’re getting a trailblazing solution from a trusted vendor. Get the details in our official announcement.

GA4 Conversion Events (Goals) Import

Imagine you’re a chef, and you’ve just spent hours carefully preparing a feast. But when it’s time to serve, you realize you’ve left all your dishes in the kitchen.

That’s what it’s like setting up conversion events in GA4 and then not being able to use them in your A/B testing tool. But with Convert Experiences’ new GA4 Goals Import feature, that’s a thing of the past!

Just like a waiter whisking your dishes from the kitchen to the dining table, our GA4 Goals Import feature allows you to easily import the GA4 conversion events you’ve set up within Google’s ecosystem into Convert. This means you can gauge the success of your variations and experiments against these goals, all within Convert Experiences.

What’s better: This feature is packaged within our one-click integration that requires Google authentication. It’s like having a personal assistant who not only brings your dishes to the table but also does them afterward!

That way, you can move from observation to knowledge even faster. Convert’s integration with GA4 gets even better…

GA4 Automated Revenue Tracking

It’s common to find your A/B testing tool reporting one revenue value while Google Analytics reports another. That’s not a good look when we’re talking about money.

With Convert’s new GA4 Automated Revenue Tracking feature, that’s over. Our tool will use the total revenue collected by GA4 as the basis of its own revenue reports and Average Revenue per User (ARPU) calculations.

This means you’ll get trustworthy results that align with your GA4 data, saving you hundreds of hours of troubleshooting and test design reviews. Yet another way we help you get from observation to insights with speed.

27-Second Google Optimize Data Importer in Convert Experiences Tools Extension

We know you haven’t forgotten Google Optimize yet. It offered a low barrier to entry for many testers but also encouraged a lot of random, one-off efforts.

Our goal at Convert was to comprehensively replace Google Optimize—for the eager experimenter—with a solution that doesn’t require as many hacks, won’t break the bank, and will be there for the foreseeable future. (In fact, one of our first customers is still with us — paying $9 per month).

So, we’ve built a data importer that allowed you to quickly and easily migrate your Google Optimize data into Convert Experiences.

All of this baked into the Convert Experiences Tools Chrome Extension, which also helps you optimize your experience development, testing, and debugging through some key features.

The extension gives you deep insights with detailed logs, instant access to Convert’s installation snippet, and smoother workflows with the Visual Editor.

You can manage experiments directly in your browser, make code changes on the fly, verify your target audience, and troubleshoot conversion issues with ease.

Convert Assistant

It’s a custom GPT you can access via ChatGPT with current knowledge of our extensive support database and articles. You no longer need to sift through pages of documentation or wait for support responses. Simply ask and receive.

To use Convert Assistant, you need a ChatGPT subscription. It allows you to use GPTs. 

Then, you can reach out to us via support@convert.com or our other help channels for direct access to the GPT. Because it isn’t publicly available yet.

Then, whenever you’re curious about a specific feature or how to optimize its use, ask the Convert Assistant GPT. It will give you specific answers to your questions in seconds.

Facing an issue or need guidance on a particular topic? Convert Assistant taps into our support documents and blog articles, offering you precise knowledge in real time.

Beyond Convert-specific details, the Assistant is well-versed in the broader realm of experimentation. Whether you’re new to the scene or a seasoned pro, it’s like having a quick guru by your side.

Conclusion

In lieu of a conclusion, we’ve drawn upon insights from Ton Wesseling and Jonny Longden’s thought-provoking LinkedIn posts, which provide guidance for selecting a suitable experimentation tool vendor.

So, if you’re wondering what to consider when making such a choice and if Convert is the right choice, here’s what you need to know:

1. Pricing Structure: Understand the pricing structure of the tool. Is it cost-effective for your organization’s needs?

Convert provides flexible self-service pricing. Plus, we don’t force you into an annual plan and our pricing is significantly more affordable compared to what’s popular out there. On top of that, we give you a free 15-day trial with no commitments.

2. Integrations: The tool should have the ability to integrate with other platforms, particularly data integrations. API capabilities are also important.

Our API V2 is out and we’ve got over 90+ integrations with tools in your martech stack.

3. Support Availability and Quality of Documentation: Good customer support and comprehensive documentation are crucial for troubleshooting and understanding how to use your experimentation tool effectively.

Besides our comprehensive support resources, you can reach our responsive support team from inside our platform in just 2 clicks.

4. Data Quality: The tool should provide high-quality data, including SRM (sample ratio mismatch) aspects.

Convert Experiences comes with a useful SRM checker. Once enabled, the tag will look like this:

5. Transparency on Statistical Models & Ability to Run Your Own Stats Calculations:

The tool should provide clear information about its statistical models to ensure trustworthiness. It should also allow you to perform custom statistical calculations.

We’re very open about a lot of things. This one is no exception: Here are the statistical models we use.

Plus, Convert’s A/B testing significance calculator supports revenue metrics like AOV & ARPV.

6. Server-side Solutions: The tool should offer server-side testing capabilities. Our full stack testing feature allows you to test both front-end and back-end aspects of your product. We support both server-side and client-side testing.

7. GDPR Compliance and Strict Cookie Policies: The tool should be compliant with GDPR and other relevant data privacy regulations. We’ve been the forerunner of data privacy, security, and compliance for many years now. We’re GDPR and CCPA compliant.

8. Ability to Influence Product Roadmap: The tool should allow for user feedback and influence on its product roadmap. We maintain close relationships with our users and agencies who use our product. We listen and use your feedback to evolve Convert.

9. Targeting and Segmentation Functionalities: The tool should offer robust targeting and segmentation options.

Convert offers an advanced targeting engine with 40+ stackable filters, and the flexibility to use data locked in 3rd party systems. Plus, after your experiment starts receiving traffic, you can access the advanced post segmentation feature:

10. Load Time / Flickering: The tool should have minimal load time and prevent flickering issues. We have super fast loading times and solved the flickering problem years ago, back when it was still a luxury.

Anti-flicker technology has always been baked in our tool at no additional cost.

11. Ease of Use / UX: The tool should be user-friendly and manageable, even with multiple accounts/tests. We have countless reviews praising our “idiot-proof” UI.

12. Consistency and Predictability of Traffic Volumes: The tool should be able to handle consistent and predictable traffic volumes.

14. Visual Editor and Code Editor Feature: The tool should provide both a What You See Is What You Get (WYSIWYG) editor and a code editor.

Convert’s robust Visual Editor is usually one of the first features our users fall in love with — and the code editor features give you extra flexibility in building tests with custom JS and CSS.