Content Marketing – Content Experiments

A/B Testing – Data Makes The Call!

“In God we trust. All others bring data.” This is a quote often credited to W. Edward Deming but I have found no data to indicate that he is the originator of this. It is more likely to have come from the book “The Elements of Statistical Learning”.

The point is that there are content marketing development tools that can help us to improve our content so that it is more effective at what we want it to do.

A/B testing was once a classic direct mail tactic. However, A/B testing is now firmly rooted in online content marketing to test variations related to online ads, e-mail campaigns, website pages, landing pages, or even entire websites. A/B testing is a fundamental tool to help content marketers improve the effectiveness of their content with the intent to increase the “delight factor” for target audiences.

Over the past few years A/B testing has become a high-stakes secret weapon of content marketing and web page conversion. Conversion is having a visitor to your website achieve an objective. This may be picking up the phone to call you, subscribing to your newsletter, filling out a poll or survey or making a purchase. Conversion is an objective of a successful content marketing program.

What is A/B Testing?

Web Page Conversion - A/B Testing Let’s take webpage conversion rate as an example. When you are interested in making a change to improve the web page conversion rate of any of your website pages you may proceed with some uncertainty. This is natural. However, you have to conduct content experiments to determine if your web page conversion is as effective as it could be.

Do you run experiments? There is risk to this. You could make things worse or some other factor may skew the results.

A modern A/B test is a way to “focus-group” test in real time. Without being told, a fraction of the visitors to your website page are diverted to a different version of the page (the experimental content) and the web page conversion performance is compared against the visitors that are presented the standard page (the standard content). If the new version is superior – creates longer visits, more purchases, more sign-ups – it will displace the original. If it is inferior then you will simply stop using it and design another version of the content.

An A/B test allows seemingly subjective questions of design – color, layout, image selection, call-to-action – to become incontrovertible matters of data-driven social science.

A/B Testing = Competition Beating Activity

If you have traffic reaching your site and have something to offer that you think is delightful and valuable A/B testing can be a content marketing game changer. Here are some clear advantages:

It can eliminate time-wasting meetings trying to decide what is best. Let the data speak! Don’t struggle to figure out the best phrasing for an important headline – run all the proposed headlines and let the A/B testing decide.
It can help you to decide on which are the best improvements.
It helps you to make content marketing decisions with confidence. You know, with data, what works and what doesn’t.

A/B Testing – Some Cautions

One could consider this a caution or a weakness but it is a bit philosophical. The issue is that often, with A/B testing, we do not understand the “why”. We know what works and what doesn’t but we do not know why. A content marketing approach seeks to learn the reasons why but perhaps it doesn’t matter. If you can improve the effectiveness of your content marketing, you can worry about the “why” later.

Another caution is that an A/B tester tests only the little things that affect performance. Small changes are perfect for A/B content experiment testing. However, I refer to this as trying to find the local maximum (remember your Calculus I). Once you conduct an A/B experiment and it is worse (perhaps just by a little), you turn back. However, do you have the nerve to make a big change and find the global maximum? Small A/B testing experiments may protect you from a major negative change but you need to also consider how costly it may be for you if you never find the most optimized version.