Predicting Direct Mail Test Results Before You Mail. Not A Theory. It s Proven. Scientific Survey Pre-testing for Direct Mail Marketers

Transcription

1 SPEC I AL REPORT Scientific Survey Pre-testing for Direct Mail Marketers Based on Bayesian Analytics Predicting Direct Mail Test Results Before You Mail Not A Theory. It s Proven. By Gary Hennerberg Author, Business Marketing Analyst & Strategist

2 Page 2 Predicting Direct Mail Test Results Before You Mail Up to 256 Variables Pre-tested 95% Statistical Confidence... All For About What You Would Spend for 3 Live Direct Mail Tests A confluence of technology and something called Bayesian Analytics now enables direct mailers to pre-test and predict future response before mailing. Bayesian Analytics may well upend how we test to identify the highest profit-producing control faster and at a fraction of the cost of traditional testing methods. Bayesian Analytics is already being used in astrophysics, weather forecasting, insurance risk management, and health care policy. And, now, a few cutting edge mailers have successfully used this analytics approach, too. So what if you could predict the response rate of your direct mail test before you actually printed and mailed it? You walk a plank every time you test in direct mail. It s expensive. Offers and creative are often tested based on gut feel, and the reality is that the odds of beating a longtime direct mail control are low. With a new and proven methodology, you can now test as many as 256 variables with a 95% level of statistical confidence and get key insights in advance of mailing. Traditional Direct Mail Testing At first glance, you might think this is just another multivariate test, where you create several direct mail packages and test them all at once. That s not how this technique works. Currently when you test using a multivariate methodology, you probably consider testing a couple of different format ideas, two copy decks and a handful of offers. This process allowed you to test many variables all at the same time, but were limited to about elements. New creative is developed and you determine test quantities. When performing a multivariate test you always like to have test cell quantities of at least 5,000 pieces. Total quantity for your 15 test packages is 75,000 pieces. Likely total cost of this effort: $15,000 - $20,000 for creative development of 15 new direct mail packages, plus over $55,000 for printing, postage and rented lists (assuming $700/M, higher than a roll-out because you re printing in smaller test quantities). >KEY FINDINGS With a new and proven Bayesian Analytics based methodology, you can now test as many as 256 variables with a 95% level of statistical confidence and get key insights in advance of mailing.

3 Page 3 Total cost of traditional multi-variant direct mail test: around $70,000 or more. Under the scenario, it did help many companies find a new control faster than using their normal testing process, but it was still expensive and didn t give you as much intelligence as this new test methodology based on Bayesian Analytics. off. And you want to test multiple headlines. And images. And formats. This matrix more visually illustrates the complexity of testing multiple variables. Let s say you want to test four different pricing offers, four headlines, four imagery graphics and four direct mail formats. Multiplying 4x4x4x4, there are a possible 256 test combinations. Modern Direct Mail Testing This new test methodology tells you what mix of variables will deliver with 95% statistical confidence a predictable future response rate. It s the ultimate in predictive analytics with this twist: predictive analytics typically are based on past response. Now you can mail based on how a prospective customer will respond in the future. And you ll understand why a specific combination of variables worked better for specific demographic and psychographics. Usually, direct mail marketers test four categories of variables. For example: 1. Price 2. Headlines 3. Imagery 4. Formats Within each of those variables, direct marketers often want to test even more options. For example, you might want to test $5 off, versus $10 off, 10 percent off versus 15 percent When you multiply four offer tests (left column), across the four testing categories, (across the top), the total possible combinations are 4x4x4x4 to equal 256. Of course, you can test fewer attributes if desired (it s not recommended that you test more, although multiple studies can be done in sequence). In traditional direct mail testing approaches, it s impractical and costly to test 256 combinations. Even if your response rate dictated you only needed to mail 5,000 volume per test for statistical reliability, you d still have to mail over 1.2 million pieces. If each piece costs $0.50, the total cost is $600,000. Not only is it cost prohibitive, in our dynamic culture, market changes could render your results obsolete in only a few months. >ELIMINATE SPECULATIONS Bayesian Analytics methodology takes the guess-work out of what to test in a live mailing scenario.

4 Page 4 Not a Theory. It s Proven. Knowing there must be a better way to affordably test multiple variables, this new pre-test methodology offers a cost-efficient breakthrough for direct mailers. Bayesian Analytics works with a fraction of the data required to power today s machine learning and predictive analytics approaches. It delivers the same or better results in a fraction of the time. Applying a Bayesian Analysis methodology, direct mailers can make big and statistically reliable conclusions but from less data. Bayesian Analytics frequently produces results that are in stark contrast to our intuitive understanding. How many times have you used your intuition to test a specific combination of variables thinking it would result in a successful direct mail test, only to be disappointed in the results? Bayesian Analytics methodology takes the guess-work out of what to test in a live mailing scenario. Instead of testing and guessing, you can now pre-test those 256 combinations of variables before the expense of a live test mailing. The pre-test reveals which combination of variables will produce the highest response rate in the live test, a substantial test savings. In test after test, this system has reliably forecast future response rates from a variety of variables. It s not a theory. It s proven. Here s how it works: You provide a customer profile or a list of current customers to identify audiences. Testing variables are determined. A panel of prospective recipients who have opted in as online survey participants are identified. These individuals are vetted and compensated to ensure both quality data and a high level of participation. Your test is presented to the prospective customers, but first they answer a few questions about themselves. If their demographics or psychographics don t match your criteria, they are thanked and not permitted to continue. Prospective customers who pass the criteria are presented images of a variety of combinations. They don t actually see all 256 unique combinations! They only see about 20 images statistically positioned within about 16 questions, and are asked to tell us which images would prompt them to respond. Their answers provide us with enough information to statistically project the best combination of variables to yield the highest response. >KEY FINDINGS With a new and proven Bayesian Analytics based methodology, you can now test as many as 256 variables with a 95% level of statistical confidence and get key insights in advance of mailing.

5 Page 5 How You Predict Future Results Once the responses are compiled, you can mix and match all the 256 variables you tested. Sort the data by demographics and psychographics and two things are revealed to you: 1. What the response rate will be for any of the combined 256 unique combinations. Having this much insight allows you to understand not just which package will pull best but what it is specifically that is influencing the responders. 2. What mix of direct mail variables will achieve the best results for any of the tested demographics or psychographics. It s possible to learn that a certain set of variables work more successfully for people who are, for example, age 60+, versus those age This means you may be able to open up new prospecting list selections that previously didn t work for you. All of this before you drop a single piece of mail. Getting Started Without Risk Now that the beta testing has been completed and the methodology is proven to be reliable, now this is the time to explore this Bayesian Analytics pre-testing method. If you re new to predictive science, kicking off a new initiative, or exploring new ways to position predictive analytics at your organization, contact Gary Hennerberg today. Learn how other direct mail marketers are using predictive analytics to: Test mail with more confidence knowing your offer, package format and message are on point. Increase the odds of beating a control. Lower direct mail marketing expenses. Identify higher responding package attributes before you mail. Increase leads and sales. 073HPDM Gary Hennerberg Gary@Hennerberg.com Hennerberg.com About Gary Hennerberg Gary started his direct mail career in Today he develops custom formulas and analyzes both direct mail and digital channels. In addition, he is a successful copywriter and author of Crack the Customer Mind Code and Direct Mail Quantified: The Knowledge is in the Numbers. Gary is sought-after for his integration of direct mail, catalogs, marketing, websites, content marketing, search marketing, re-targeting and more. As Seen In: Direct Marketing Quantified The Knowledge is in The Numbers By Gary Hennerberg Crack The Customer Mind Code Seven Pathways from Head to Heart to YES!