From real estate to the stock market, to energy, to technology, to politics to fundraising the Pandemic/Supply Chain/Ukraine/Inflation Virus is wreaking havoc.

This is certainly the case where direct response –both direct mail and digital—is concerned. Massive shortages of envelopes and paper with prices going up and availability still down. Significant production delays and bottlenecks.  Postal rates increasing between 6% and 8% next month. And digital platforms/ channels choked with extremist and grifter sewage occluding the flow of legitimate advocacy and political.

Thus, this post for smart fundraisers willing to brave fear and turmoil while the timid freeze or limit fundraising spending – and of course suffer the inevitable declines.

We’ve written over and over about ways to limit risk, reduce costs and significantly boost results in tumultuous times.  By reporting on the results of approaches, tests and proven results of the behavioral scientists and fundraisers at DonorVoice.

Today we’re spotlighting  a remarkable time and money saving tool created by DonorVoice  and introduced in in the Agitator Toolbox 10 years ago. Over the last decade it’s  proven its value and speed.  It’s known simply as the  Pre-Test Tool

We introduce it with the headline “18 Months’ Worth of Testing in a Day” because instead of doing time-consuming and incremental testing you can test hundreds of variables at once.  You can get a detailed description here or check out the 2-minute video below. The video was originally created for direct mailers, but the Tool works equally well for digital applications.

Why the Pre-Test Tool is So Valuable –Especially in These Times

Given the horrendous time and cost constraints of the direct mail supply chain and flood of digital overkill here are some reasons why smart fundraisers will check out the Pre-Test Tool.  (And if you have questions and or/want to discuss how you can put it to work just shoot Kevin and email and he’ll fill you—kschulman@thedonorvoice.com

• The direct response test ideas that actually make it into the mail digital stream are almost all incremental and rarely (mostly never) beat the controls.
• Enormous amounts of time and money are spent coming up with test ideas, producing them and managing the logistics, only to have the same poor results year over year.
• The more creative or innovative ideas tend to get discarded out of fear and the need to stick with the mostly known, if lousy, results of the control and the incremental changes to it.
• Even if you do manage to test big ideas, the results are muddled at best since you changed EVERYTHING, instead of one element, and while there may be many good ideas in the creative package, they are forever lost amid the bad ones.

How It Works

• The pre-identification of likely winners and losers is done in two parts:
• 1)First, surveying a representee sampling of your own donors  or in the case of acquisition target groups who will receive the actual mailing, showing them visuals and copy of the and measuring preferences using a very specific and battle-tested methodology. (For simplicity’s sake think about how an optician uses and eye chart.)
• 2)Next, using the data to build a statistical model to assign a score to every single element that was evaluated.
• This methodology is well established and used by large, consumer companies (e.g., Coca Cola, General Mills, Proctor & Gamble) to guide product development for many of the sodas, cereals, and detergents on grocery store shelves.
• Over the past 10 years Kevin and the DonorVoice crew have transformed that process into as effective solution for nonprofits that is as close as that tube of toothpaste in your bathroom.

The Importance of Avoiding the “Baby and Bath Water” Problem
in Direct Mail Testing

• One particularly aggravating problem in most direct mail testing is throwing out the baby with the bath water. This happens when the organization sends a test package with numerous test elements – i.e., a whole bunch of stuff is changed.
• The results for the direct mail package or e-mail are a very crude measuring stick for performance, only giving thumbs up or thumbs down for the entire package, with zero guidance as to whether individual components were well received (i.e., the “baby”) even if the bath water needs to be changed.
• This happens all the time. The only alternative, which as a general rule, NEVER happens, is to deconstruct the totally new package into a series of A/B tests, with each test panel only including a single change. Even if this were done, it would take forever and a day to execute. Certainly, some groups may try to read the tea leaves and infer or guess — based on years of experience and past testing — about why a package did poorly and what might be salvageable. But clearly, this is a flawed process fraught with layers of personal bias.
• There is a better, empirical way. For the past 30 years product developers have used a survey-based methodology to identify and separate the the baby from the bath water. This process can be done in days versus months, costs a fraction of what traditional testing costs, and like a client told Kevin, is “like doing 18 months of testing in a day”.  (To learn more about the methodology, click)
• Here is a recent example of theold-fashioned way. Client X mails a totally new package – different OE/Subject Line/Graphic, letter/message format, copy, inserts or landing pages – against the control. New package performs poorly. Money is lost. Time is lost. New package is thrown out.
• The better way. The New Package is never mailed, because the client pre-tested it (along with hundreds of other package combinations) and determined it would not perform well, as constructed, against the control. Thousands of dollars are saved, many insights are gained that would have taken years to accumulate with conventional A/B Testing, and the cost of the pre-test is more than covered.
• Most importantly, a new ‘baby’ or package element is potentially discovered when two components of the new package test quite well (as determined by an actual score assigned to every single element). These New Elements are now live tested in the control package and replace elements of the control that are identified as weak.
• New “bath water” (i.e., poor performing test elements) are unfortunately easy to create; but “baby elements” (i.e., winning package elements) are much more difficult to identify. But in the case of the Pre-Test Tool you can determine which elements of an appeal or prospect campaign work best – Teaser/Subject, message copy, graphics, signers, response form/landing page, asking amounts, whatever elements you choose.

Given the frantic, uncertain climate we all  face isn’t it time to start identifying the winning “babies”?

Roger

P.S.  A couple of notes on digital vs direct mail and why properly constructed testing is so important.

Note 1: What works in direct mail doesn’t necessarily work in digital. Check out this post. Paper v. Digital: Does the Medium Matter?  Without question you’ll raise more money with PAPER.  A top reason why the Pre-Test Tool is so valuable in saving time and money.

Note 2.  Do you think that maybe, just maybe, the proliferation of emails and its almost concomitant hokeyness of faux countdown timers, matching whatchamacallits, gimmicky, cutesy subject lines is undermining credibility and seriousness? Wouldn’t you like to know –in advance –what parts of your message, offer, calls to action punch through?

Note 3.  For answers to your questions Kevin’s email is kschulman@thedonorvoice.com