Drive Certainty in Uncertain Times With Experimentation

Just gonna go straight for the elephant in the room: COVID-19.

It’s had a wildly transformative effect on our current daily lives. Our carefully crafted business plans have been thrown out the window.

If you’re in exercise equipment, toiletries, dog food, or bread makers, maybe you’re overwhelmed trying to meet all the new demand.

If you’re in travel, swimwear, food service, or public events (sports, movies, concerts), maybe the bottom has fallen out and now you’re struggling to make payroll. Point is, you’ve suddenly had a bunch of additional challenges dumped on your lap and you don’t know what to do.

Lots of marketers are throwing around phrases like “uncertain times” and the “new normal”.

What do these mean? Is uncertainty now normal?

In a sense, yes. Things are changing rapidly. What you “know” today may not be true next month or even next week. How can you possibly make any decisions in an environment like that? You don’t know. Trust your gut doesn’t work anymore.

Things are changing rapidly. What you “know” today may not be true next month or even next week. How can you possibly make any decisions in an environment like that? Start an experimentation program. @convert Share on X

Now is the time to start an experimentation program. The key point here is program. Develop a testing plan and process; no shotgun approach or “test everything”. Don’t dive into the deep end before learning how to swim. Figure out what works (and stay away from what doesn’t) by applying the scientific method to your decision-making process.

1. Create a testing plan by doing research. Come up with a set of questions you need answered. Propose solutions and how success will be determined. Something like: “Do we need to add a chatbot on the website to help cover the increased support load?” Or, maybe you’re in the opposite situation: “can we afford to keep the chatbot since revenue and traffic are down over 60%?”. This is the foundation for developing a hypothesis (tip: use a hypothesis builder to validate your ideas).
2. Sort your hypotheses using a prioritization method like the PIE framework, ICE scoring, or PXL method (or come up with your own method that’s relevant to your unique circumstances). If you’re a Convert user, check out this resource to figure out how to sort and filter your hypotheses.

3. Conduct experiments starting at the top of the list and work your way down; reprioritizing as new ideas get added.

Not convinced yet? Good. A healthy dose of skepticism is a good trait for an experimenter to have.

Those social media brag posts showing a 250% uplift? Yeah, that’s probably BS. What were the test’s run parameters? Did it have an adequate sample size, run time, and conversion thresholds? Was the experiment served to the right audience? Probably not.

Experiments need to be backed up with statistical methodology.

That’s where the certainty comes from; insights backed by data and rigor. Ignoring the statistical side is no better than trusting your gut, and we already said that wasn’t working out.

Being stuck in a test-by-test approach isn’t yielding any significant results either. Short-term success can trick optimizers into never putting an actual optimization program in place. That’s when you get a temporary uplift in winning tests, but no data to replicate in future tests. That’s why building a solid experimentation program is essential.

By creating a testing plan, developing and sorting your hypotheses and digging into the data you consistently to reprioritize, you can create a replicable system that delivers consistent results, even when everything else in your business seems uncertain.

You have questions, but also have the power to find your own answers. Start building that experimentation program today.