- Melbourne, Australia
- FAQ
- About AML
- The List Report
- Contact Us
By way of continuing on the theme of “testing”, I was reading an article during the week where a few of DM’s guru’s gave some insight into different aspects of testing. Let me share them with you:
“Two rules and two rules only exist in direct marketing: Rule #1, Test everything; Rule #2, See Rule #1.”
—Malcolm Decker
This one speaks for itself and is the mantra for most Direct Marketers. However, many marketers who coming into this space find the notion of testing harder to justify. I recall with great fondness my time working in television where an advertiser would choose smaller markets like Adelaide or Canberra to test new creative for their TV Commercials. In essence this is no different, but DM is so instantly measurable that perhaps there is a fear factor with testing. What if the results are so bad that the rest of the campaign is shelved?
“Don’t test whispers,” said the late guru Ed Mayer.
By that he meant do not test $19.95 vs. $19.99 or pink paper vs. blue paper. Testing is expensive and should be used to find breakthroughs. Whole new sets of creative, substantively different offers and even new media channels should be tested to ensure what you say, how you say it and what you have to offer can be measured and assessed.
“Confine your tests to important changes, not trivial ones. You will never get the exact same answers over again.”
—Maxwell Sackheim
Testing is not about determining which point will work best, it is about generating new response levels by doing things completely differently. What if scenario’s need to be developed and business model changes are things that demand to be tested before any roll out can be considered.
“The Holy Grail of direct marketing is the single-variable test.”
—Don Nicholas
Just like any scientific exercise, the complete change of the control will not yield meaningful learning. You have to look at one element at a time to determine the way forward. Test creative changes, then test an offer change, then test a channel change and then finally test a data change. If you mix two many elements into a test then the results may not be helpful in determining the best strategy for roll out campaign.
“Let your competitors test for you,” wrote Anver Suleiman.
Don’t test things your competition has already tested, especially the big competitors. If they tried a certain approach and dropped it, assume it didn’t work. And if they are using something over and over, assume it does work. No need to retest or reinvent until your model proves itself. I mentioned in a previous DM Tip the value of seeding yourself with your competitors to receive their DM communications, even if this means that you have buy one of their products to get on their database, it may be the best investment you have ever made. Avoid using DM communications that look too similar and also make sure that you are not trying to re-invent the wheel.
“If you make a mistake, call it testing; if you make a big mistake, call it research.”
—Wendell Forbes
Any form of marketing is a learning process and as such mistakes will be made. Marketing requires the resilient and patient commitment to achieve the desired objective. I have had to advise many clients that if they do not have the patience or the courage to undertake a direct marketing campaign, then DON’T try a make or break promotion, this will only end in tears!
“Never test in a market that cannot be rolled out in larger volumes.”
—Bill Joseph
This is the curse of Australian Direct Marketers. In so many cases we have small universes that will mean that test cells have to be small and we have to get it right because the universe will also be small. My recommendation is that even in the test mode (especially if it is an acquisition campaign) use broader rather than narrower selects so that the roll-out universe is as large as possible.
“For a test cell to be a clear winner, the number of responses must be more than double the square root of total orders.”
—Axel Anderson
Now we are getting technical and YES DM is a science...if you need help with this then ask!
“Guessing at results is as pernicious as misreading them.”
—Martin Gross
DM has little or no room for guess work, if you don’t know then you have not set the testing matrix up properly, it is as simple as that. Testing is as scientific as the genetic research. You monitor and measure every element and then make decision based on the analysis. It is a painstaking and detailed process that requires an eye for detail and, YES, that word again - patience.
Always test away from a control slowly and continually back-test until not a shadow of a doubt exists about the worthiness of the new control. The essential message is that your DM activity needs to be organic and fluid to the point where there is a constant test, learn roll-out and then re-test cycle. Marketing can be a science and can be treated as such, it just takes - yes, that word again - patience, as well as commitment and courage.
Posted on 30th June 2008