TESTING: Go Beyond Individual Communications
When you want to find out if your control package could be beaten, you test a different communication against it.
So how do you test if your direct marketing program could be better? Clearly, you test a different program against it.
For some this is a scary thought: it’s hard enough to deliver on one program effectively. But frankly, this is the only way to test the fundamental assumptions at the heart of your program:
Am I communicating too much? Roger highlighted a program-wide test by the Union of Concerned Scientists last year. The full post, and the video below with Laurie Marden, are well worth watching.
The TL;DR version is that UCS tested 12-15 appeals for one panel of 25,000 and four (yes, as in the number between three and five) to another panel of 25,000.
They received an additional $8,000 in net revenue with the four appeals and this reduced cadence became their control.
They aren’t the only ones. Catholic Relief Services tested a pilot program that cut 6-7 mailings and had a reduced email diet emails to a select pilot group. The results are here and that program is now their control program for 2018.
Am I asking the wrong way? The US Olympic Committee saw much of their donor file was premium dependent. For a $20 donation, a donor could get all manner of Olympic swag in the mail. Not surprisingly, this attracted more donors interested in the merchandise than a philanthropic gift to support Team USA.
Then…USOC tested a bold step: changing the offer to focus on a philanthropic gift – specifically, asking donors to join the Sixth Ring, a group of donors who give $100-plus per year, whether in single or recurring gifts.
And they tested this in parallel to their existing premium program, finding a 27% increase in net per piece (you can read Roger’s full discussion of this here). This, too, is now their default offer.
I’ve run a premium-dependent program before; the premiums become like crack: they give you a quick hit on results. So, you feel you need to keep using them. But. the results winnow over time, and you drive away those people who aren’t interested in premiums/crack. (Or so I’m told. I don’t even know how to use baking soda in a cake.)
I found I couldn’t break out of premium-dependent by testing just one piece at a time. Like USOC, it takes testing an entire pilot program to see a world without premiums.
What if my communications to a group of people are off? Tom talked about the idea of the long-term test back in 2011. Specifically, he raised the issue of split-testing new donors to an organization and testing different versions of welcome and first-year communications to see what maximized retention of quality donors.
This is truly the only way to test the effectiveness of your intro communications during that critical first year (where most of your donors will leave even if you are doing well). We’ve preached soliciting feedback from your donors immediately, including commitment, satisfaction, identity, and preference information. That’s a tall order.
Should it be in one giant communication? (No)
Two communications over time? Three? Four? (Perhaps)
Fifteen different communications over the course of the year each asking for a separate piece of information? (Definitely not)
The point is that your initial instinct may not be what best gets you the information you need to market effectively.
Because each of these questions cuts straight to the assumptions that underlie your program, with outcome measures like retention rates and long-term value, they can’t be measured with a one-piece response-rate-times-average-gift approach.
Creating a pilot program – one that tests a transformative variable across communications and media – is the only way to get at these deeper questions.
Have you done this in your program? Can you share results?
Nick
4 responses to “TESTING: Go Beyond Individual Communications”
Behavioral Science Q & A
Integrating an individual giving appeal with other communications from a charity can have both positive and negative effects, and the outcome largely depends on how it’s executed. Advantages of Integration Brand Consistency: Maintaining a consistent appearance and messaging across all communications can reinforce the org’s brand identity and strengthen brand recognition and trust among your […]
Read Full Answer
I’m not aware of any in-market tests specifically comparing recurring vs. gift frequency language. I suspect the answer might not be the same with all gift frequencies, nor with all people. It sounds like a great opportunity for you to test and find out what works for your audience. Based on the literature, here’s a couple […]
Read Full Answer
Based on what we know from existing data, those renewal notices can actually be pretty effective in getting people to donate. They tap into our psychology – creating a sense of urgency, reminding us of past support, and using personalization to make the message hit home. They’re playing on our natural tendencies to feel obligated […]
Read Full Answer
Interesting question. I had a quick look at the testing done on this topic. On the positive side, in all cases, over half of donors decide to cover the fee. In some cases, it goes as high as 65%. Not a negligible percentage at all. Here’s another test from iRaiser showing consistent results (see point […]
Read Full Answer
There’s just one thing to consider when designing a supporter journey: the supporter. More specifically, you need to take into account: Who the supporter is i.e. their identity, which is the reason they support this cause, and their personality, which describes the way they “see” and process the world. These will determine the kind of […]
Read Full Answer
I’m not an expert in this but a quick search surfaced this article on the effect of tax reforms on 2019’s charitable giving. The researchers didn’t find a reduction. Actually, they observed an “increase in charitable contributions in 2019, even with the lower tax rates and the dramatically smaller number of taxpayers who itemize their […]
Read Full Answer
hi, love this research Nick, thank you so much! BUT I know that not all Agitator readers are big nonprofits.
Just one thing to remember, many many many small to mid-size organizations are not at the stage where organizations like Catholic Relief Services, Union of Concerned Scientists and Special Olympics are in terms of number of mailings a year.
Most organizations I see (and I always ask this question at my webinars how often do you ask for money): typically send out one or two appeals a year with additional event invitations. Some have donorcentric newsletters with a reply envelope in them but most don’t even have that.
I’ve worked with some organizations to add one or two more appeals and every single time this generated a lot more money for the organization… Let the statistics show that for one homeless organization adding two appeals generated 32% more revenue… that’s a big deal for a $500,000 organization! Without donors being alienated, mind you!
So, it’s important to start with the current situation: how often do you reach out to your donors now, what’s the average gift, response rate, what can you do to generate more money while treating the donors with respect… for many there’s room to grow by adding, while for others it means there are ways to eliminate appeals…
Just something to bear in mind. What’s your starting point? What’s your current communication stream?
Here’s the reality: if you bring in new donors at the end of the year and then they don’t hear from you for another year… what do you think the retention rate will be?? LOW, right?
cheers, Erica
Erica, an excellent point. I would argue that nonprofits that send 1-2 appeals per year should likely test whether they are communicating too little the same way an organization that is sending 30 appeals should test whether they are sending too much – by running two separate programs where one gets fewer communications and one gets more.
This does present greater challenges for a nonprofit with a smaller list, but it’s well worth the test because, as you say, most donors will not object and may embrace a third mailing in a year.
One cautionary (and probably obvious) note: I was working with an organization that did only one appeal annually and was questioning whether to add a second one. I asked about results; they said the first mailing broke even. My advice there was to fix the appeal they had before looking to add a second one. 🙂
One more cautionary note to all charities, large and small but especially the latter who have yet to witness the massive diminishing returns of the large groups.
Don’t make the mistake of thinking the formula for success is: ask=give. It is only slightly less incomplete to think the formula is “ask well=give”.
Asking is a necessary precondition. Asking “well” improves on the likelihood though the definition of “well” is often reduced down to generalities (use emotion, certain pronouns, make it about them, not the charity).
But, even if ‘well’ is defined and delivered as donor level (not segment) understanding of motivation (i.e. identity) and need (i.e. specific interest) you will create irritation with more stuff out the door, whether it asks for money or not.
Consider this hypothetical. You are a start up charity and your goal is to get someone to give you two donations (cash, not recurring monthly) a year for 10 years. What would be your plan to achieve that?
Nobody (in their right mind) would answer that question by saying I’m going to mail them 18-24 times and send 50 emails. And yet, that is where many large charities are today. And very few are getting an average of 2 donations per donor, per year.
You need a much fuller understanding of all the (empirical) drivers of giving, most of which have nothing to do with the solicitation itself but instead all the mental preconditions, via other interactions, necessary to maximize response when the ask is made.
Ask=give is woefully incomplete in understanding the science of fundraising. Ask well=give ain’t much better.
Kevin and Nick: wow, you are good. And Kevin: that hypothetical about mailing 18-24 times in addition to 50 emails, all to winkle out 2 gifts a year? I burst out laughing. And it IS where we (including me) are today. Keep it up. Learning.