It is inarguable that increasing the number of mailings increases current demand/responses.  Send out more mail, get more demand/responses.

But, this decision making behind “mail more, make more” lives in the short-term.  We estimate the probability (usually crudely with RFM business rules) that a donor will give and then include them or not.  Because the selection rules tend towards the simplistic – e.g. recency 0 to 24 months – most donors get included most times.

But what about future demand?  Does today’s mailing make it more likely someone responds to tomorrow’s mailing?  Or, is it possible it has a negative impact on future demand?

And invariably with the Agitator and DonorVoice, the corollary question always needs to be: does the same answer apply to everybody?

A test/control study done by academics at MIT with mail order catalogs provides some answers.  We’ll give away the ending now, but the details are worth digging into.

The findings challenge conventional wisdom, which oxymoronically, is often not very wise.  Nonetheless, conventional wisdom is powerful because it tends to invoke confirmation bias as we work hard to avoid, discount or ignore contrary findings.  If that’s you, try to dig past the bias and ask yourself why the MIT findings might be real?  What circumstances would have to exist for this to be correct in your organization?  And then, read the details.

Summary of Findings

1. Lots of mailings to your ‘best’ consumers (read: donors) significantly reduce future demand.  This is because of time shifting – moving dollars forward.  All that mail doesn’t create new money from your ‘best’ donors, it moves it forward.
2. Almost all the short-term increase in giving is offset by decreases in future demand.
3. There is cross-channel substitution; as giving goes up in mail it goes down in other channels (e.g. online)
4. The findings were reversed in the case of “non-best” consumers (donors).Sending more led to more increase in future demand.

The experiment lasted eight months with 20,000 customers, half identified by past behavior as “best” and the other 10,000 as “other” and randomly assigned to test and control.

The Test group got 17 catalogs, the Control, 12.  The extra five were additional copies of previously received catalogs.

Purchase behavior included activity pre-experiment, during the experiment and after. The chart below provides the evidence to support how most companies and charities view the world – mail more, make more.

Click to enlarge

Unfortunately for the conventional thinkers the picture isn’t so bright longer term—at least where the Best Customers are concerned. Scrutiny of customer behavior in the post-test period included statistical analysis to control for other factors.  These results below show the model coefficients, but the interpretation is simple: positive sign is uplift, negative sign is decreased demand – i.e. purchases going down.

Click to enlarge

For the ‘less’ good consumers – the Others – there is uplift during the test period that persists after the test period (future demand from current mailings).  For the “Best” customers, the current increase in promotions made them worse customers in the future – they simply shifted their dollars forward.

In short, mailing everyone the same number of times results in over-investing in some and under-investing in others.  These results also provide a warning that focusing just on the immediate, campaign level results is to ignore the complete picture as is using averages, which almost always hid a lot. 

The Details

In this experiment the “Best” customers were, in fact, more profitable in the Control condition (fewer promotions) when the time horizon is extended beyond the immediate.  The “Other” group was profitable in the Control (fewer promotions) but more profitable in the Test group (more promotions).  This is a classic example of the local vs. global optimization problem of thinking a “good” result is “best” when the “best” is an option not considered.  [ To understand how failure to distinguish “local” from “global” optimization download this free DonorVoice white paper, Barriers to Growth.]

Why do so few companies or charities adopt a longer-term view?  As it turns out, unicorns do exist.  A German mail order catalog company, Rhenania, did revise its mail strategy from maximizing short term to include consideration of future profits.  As a result, they reversed their declining sales and market share.  (For some examples of mailing less, making more see this Agitator post, Less Contact, More Revenue. )

An important, nagging question remains in all this: why did the “Other” group in the MIT study behave as it did?  If time shifting of spend explains the “Best” customer behavior, why didn’t that hold for “Others”?

The Questionable Value of “Brand Building” and Informing

Mailings have two jobs; persuading (immediate sales) and informing (future sales).   Persuading in this context is not about converting the non-converted, it is more about selling those already mentally sold (Best) or partially sold (Others open to buying).

Informing can also be thought of as brand building and creating positive impressions.  One hypothesis is that the Other group got a two-fold benefit from the mailings (persuading and informing).  The MIT researchers analyzed this as well, using the number of mailings over the prior past five years as a proxy to identify those who might benefit from the Informing role (fewer historical mailings) and those less likely to benefit (more historical mailings).

Their analysis did not find support for the Informational explanation that those in the Other group got more familiar with and exposed to the brand and increased share of spend in the present and future.   The number of past mailings was not a factor in explaining the Other/Best customer findings.

This suggests the informational/brand building role of the mailings was negligible.  It was all about driving increased sales by making the ‘ask’ and that effect lingered for the group that had more opportunity to produce more sales – the Other group.

Your Best customers and donors can only buy/give so much.  More does not beget more for everybody. And unfortunately the upside is often the exact opposite of how we mail today.

Kevin