The saying: "I know that half of my advertising budget is wasted. I just don't know which half." comes from John Wanamaker, who is considered a pioneer of modern advertising, Thanks to modern methods, marketers today have techniques that show which elements of a campaign generate resonance with customers and why they are successful. One of the most promising methods here is experimental design.
What is experimental design?
Experimental design is an approach to research, testing and optimization that is used to organize data and perform statistical tests to identify cause-and-effect relationships between input and results.
In the context of a marketing campaign, experimental design allows brands to identify different elements separately and test variations of each element to see which combination is best. In this way, not only can the best version of each element be identified, but it is also possible to assess which elements have the greatest impact when combined.
For example, a branded company trying to find the best design for a website can separately mark different variations of the headline, slogan, image, and call-to-action button, and then test four or five variations of each element to see which combination will most effectively engage customers. This allows marketers to test many different variations of different page elements quickly and to scale.
How does experimental design differ from A/B testing?
One of the most common methods marketers use to test different messages is A/B testing. A/B testing allows you to randomly divide recipients into two groups and deliver one version of a message to one group and another version to the second group. The test shows which message is more successful. A/B tests are a kind of test between the champion version and the challenger version.
A/B tests show very effectively which of a small handful of options performs best, as Ron Kohavi of Microsoft and Stefan Thomke of Harvard Business School point out in their 2017 article in the Harvard Business Review. For brands that only want to test two or three variations of an element, an A/B test can provide a useful result with a small sample size. This simplicity has made A/B tests very popular.
However, the benefits of A/B testing are very limited in today's world of mass marketing and real-time communication. For example, these tests cannot efficiently test multiple versions of several page elements of a website against each other or show why one particular message performs better than another. Nor can they show why a message has caused no or a negative reaction. And this is exactly the sore point for modern marketers. Marketing results can only be continuously improved if you have insight into why some messages work and others don't. In this way, marketing is developing into an insight-driven discipline.
Similarly, brands cannot use A/B testing to efficiently evaluate multiple elements of the same message. They would have to perform a separate A/B test for each variation, which would very quickly multiply into an unmanageable number of tests. For example, four different versions of five individual elements on a website would require more than 500,000 separate A/B tests, each of which would have to be seen by a large number of people to obtain a statistically valid result.
In contrast, experimental design allows marketers to test a number of variations and then use predictive analytics to identify the combination of countless possible combinations that will yield the best results.
Experimental design in action
As an example of the advantage of experimental design, the experimental process used to determine the best design for a paper airplane will be used here. The basic elements to be tested are the paper size, the paper weight and two different design variants. This results in a total of eight versions of the aircraft.
An A/B test approach would require "aircraft designers" to compare each aircraft variant with every other variant and test how well certain variables perform against another specific variable. Experimental design, on the other hand, requires only four tests because it arranges the results on each element in such a way that it is possible to predict how they will perform in different combinations. This test design provides the ability to compare which variant achieves the best result and for which variant the best result was predicted.
Even testing paper airplanes with a handful of elements and variables can take all day. And it can easily happen that there is not enough time and you overlook the optimal combination. In marketing, people are by no means able to identify or predict the best possible result due to the enormous amount of data.
A/B tests require a hypothesis to be proven or disproved. But what if there was a combination of variables that far exceeded anything the human tester could assume?
Back to website design, for example: the result that is predicted to be most successful could be headline A along with text B, image C and call to action D. The result with the best prediction of success is the one predicted to be the best by the machine learning algorithm based on the experimental design. These options are then translated into real tests to test actual performance. In this way, the experimental design provides more insights to marketers - in less time and with less effort than an A/B test.
Increase sales with experimental design
A person can come into contact with a brand several times before deciding to become a customer. Every opportunity for communication and interaction with potential customers counts. If a brand can increase the number of contacts that respond to its marketing messages, it will also increase the conversion rate. Every interaction counts for success, so knowing how to engage customers quickly, effectively and successfully is critical.
A positive customer experience is achieved by marketers when they know which messages are successful and which are not and what the reasons for success or failure are. In website campaigns, for example, it is often the case that the impact of different messages depends on the channel. Customers can react very differently depending on the channel through which they reach the website. People who click through from an email may have a different engagement profile than people who click through from a social media ad. Experimental design allows messages to be viewed separately by channel, identifying for each channel the message with the best result and highest conversion rate. In addition, marketing can use the insights from this differentiated analysis for other campaigns.
This personalization thanks to Artificial Intelligence enables optimized targeting. Personalization through AI is a major trend in digital marketing. This is also confirmed by the current study "Digital Dialog Insights 2019", conducted by United Internet Media together with the Stuttgart Media University. Experts from the retail, service and production sectors were surveyed. 83 percent of those surveyed are convinced that personalization strategies can increase the quality of advertising messages and thus represent a real added value for digital marketing. And AI-supported experimental design is an important element in advancing effective personalization.
This article is a translation. To read the original German article, click here
___________________________________________________________________________________________
Are you facing an impossible mission in the fields of leadership, marketing, communications and/or politics?
Or are you currently struggling with the Corona crisis?
Then it is now the time to contact Mr. Campaigning. Since 1998 he has been supporting startups, global companies, organizations, and individuals. His clients range from "A" like ABB to "W" like the World Economic Forum in Davos.
Impossible is just a word for change.