charity: water Goes Mailing?, Part 2
Personalized Matching Findings
Winning execution Openness, losing on Agreeable.
The holdout group has no hard, incremental costs to service it – it’s all digital. The net or profit margin for the holdout in the winning Openness treatment is higher than the letter group margin. But, you got a lot more people “engaged” with the brand again; giving, exposure through different channel, etc. by including the mail in the mix.
So, what about digital, couldn’t this same thing be done there to ‘beat’ the one-size holdout control? Glad you asked and as usual, charity: water was thinking ahead too.
They ran this email test with the last email in a 2-week, country specific campaign.
The changes are subtle, an extra line of copy in the Openness (middle) and Agreeable (right) treatments versus Control (left). Plus, copy change tailored to match Personality on the button call out text.
That’s it. Everything else was identical.
Results?
Mirrors the mail test with Openness doing better than Agreeable but in this digital test they both beat the control albeit only directionally with the latter.
Summary
Test, test, test. That’s the sector mantra. There ought to be a predicate, think different, then test, test, test. A big part of charity: water success comes from thinking differently, it’s built into their DNA. Not only are they receptive to new ideas, they seek them out.
This testing, like all testing, had constraints. It also had success and failure. All of it contributes to the knowledge base and idea factory for charity: water. That is the real value. Inherent in this is getting beyond just format and channel as the lens onto the world. People are different, we can either market and fundraise to these differences or ignore them. Both are choices and charity: water made theirs.
What will you take from this?
Kevin
P.S. As promised, here is a link to both charity: water posts as a single case study.
6 responses to “charity: water Goes Mailing?, Part 2”
Behavioral Science Q & A
Integrating an individual giving appeal with other communications from a charity can have both positive and negative effects, and the outcome largely depends on how it’s executed. Advantages of Integration Brand Consistency: Maintaining a consistent appearance and messaging across all communications can reinforce the org’s brand identity and strengthen brand recognition and trust among your […]
Read Full Answer
I’m not aware of any in-market tests specifically comparing recurring vs. gift frequency language. I suspect the answer might not be the same with all gift frequencies, nor with all people. It sounds like a great opportunity for you to test and find out what works for your audience. Based on the literature, here’s a couple […]
Read Full Answer
Based on what we know from existing data, those renewal notices can actually be pretty effective in getting people to donate. They tap into our psychology – creating a sense of urgency, reminding us of past support, and using personalization to make the message hit home. They’re playing on our natural tendencies to feel obligated […]
Read Full Answer
Interesting question. I had a quick look at the testing done on this topic. On the positive side, in all cases, over half of donors decide to cover the fee. In some cases, it goes as high as 65%. Not a negligible percentage at all. Here’s another test from iRaiser showing consistent results (see point […]
Read Full Answer
There’s just one thing to consider when designing a supporter journey: the supporter. More specifically, you need to take into account: Who the supporter is i.e. their identity, which is the reason they support this cause, and their personality, which describes the way they “see” and process the world. These will determine the kind of […]
Read Full Answer
I’m not an expert in this but a quick search surfaced this article on the effect of tax reforms on 2019’s charitable giving. The researchers didn’t find a reduction. Actually, they observed an “increase in charitable contributions in 2019, even with the lower tax rates and the dramatically smaller number of taxpayers who itemize their […]
Read Full Answer
thanks Kevin, this is great and I’m absolutely for testing, but I can’t quite see how there is a statistical difference between 20, 6 and 2 conversions, they’re all the same.
Unless you have at least 100 responses, I don’t see how there can be a 95% significant difference.
As I don’t know how many were mailed for the letter and post card, not sure if those are significant either.
I can say that if the goal was prospecting of new donors, 1.4% and 1.6% are quite good and my conclusion would be that direct mail could also work for charity:water. And it certainly would help with generating donors who give through both channels.
Having said this, the tests are great as they are aimed at being more donor centric and testing new things and new wording especially.
I just read the new study from Adrian Sargeant on the use of certain words, being more donor centric (he calls it donorcentricity 3.0) and it’s fascinating how it can help lead to more … especially if fed by words generated through donor surveys. check it out.
https://www.philanthropy-institute.org.uk/donor-centricity-30
cheers, erica
Hi Erica, thanks as always for reading and commenting. Let me correct the comment about statistical testing for the record and so that the misconception doesn’t get repeated. There is nothing about 100 responses required nor any sample size requirement at all for significance testing. Sample size does matter in that if it’s large enough you could officially flag something as “statistically significant” even if it is practically irrelevant – e.g. a hundredth of decimal point difference in response could be magically flagged as “statistically significant” if the sample sizes were large enough. In this case, it is accurate reporting, there is 95% confidence that the Openness test beat the control. I hope the 100 thing isn’t out there in the ether.
A few other clarifications, this testing was not for new donors, it was folks who lapsed as sustainers. The Openness direct mail test did beat the holdout group, statistically. The Agreeable direct mail did not, they are statistically tied but the mail has cost attached the holdout condition does not so on a net basis it loses.
Openness on a net financial basis also “loses” since there is never any direct cost for the holdout treatment that is all digital. However, it did still produce a profit/net (albeit lower margin) and it did get more donors back into the fold.
There is lots of nuance here. As we stated in the post, there were testing successes and failures but all of it contributes to the knowledge base and idea factory for charity: water. That is the real value.
Kevin
Clearly, if charity:water had a personality, it would fit in the Openness category! Thank you so much for sharing these tests and results. I’m curious to know why, with the email test in particular, you tested subtle tweaks between the personalities. Did you want to see the effect of minor changes? Would you use the results as motivation to test bigger changes, such as subject line, headline, image, etc?
Sarah, nailing it on the Big Five, yes, it would definitely be high in Openness. Good question on the small tweaks. This email test was charity: water working from the DonorVoice “crib sheet” and having been exposed to all the discussion and rationale on the letter copy – i.e. the process became de-facto training for charity: water. And they own the Personality Tags on their house file for permanent, ongoing usage.
So, they made these changes and were a bit unsure if they were doing it right and therefore, wanted to make fewer vs. more in case they were off the mark in the tailoring. We gave them high marks on the tailoring and expect they’ll do more testing as the learning and growing continues since we know those Open folks love learning and have a growth and continuous improvement mindset.
Makes a lot of sense. Thanks, Kevin.
Will charity:water be rolling out with a direct mail program based on these results? Or will there be further testing? I’m curious about what their next steps will be!