Is the Donor in Your Test Design? 

July 16, 2021      Kevin Schulman, Founder, DonorVoice and DVCanvass

Most testing in our sector looks like this:

Test

Control

Object : online form, letter, envelope

Your idea to change the object

Object stays same

 

Let’s say your Object idea is to change the messaging by emphasizing the person being helped is hardworking and diligent. The message goal is avoiding people incorrectly assuming the beneficiary is “lazy” or “needy” and the donor not giving as a result.

The implicit assumption is this test idea is more powerful and important than any differences that exist among the people within the test group.  Said differently, with this test design we assume all people are the same .

Most of these types of tests fail.  They don’t beat the control.  Often this is because all people aren’t the same. The test idea was well received by some and disliked by others.  Those two groups get blurred together into our ‘everyone’ is the same approach that lumps them into one bucket.

This design is not donor centric.  It can’t be if want to raise the bar on the term “donor centric” and have its definition premised by what we all know to be true– people are different.

Compare the above test design with this one.

Donor Personality

Test

Control

Conscientious

Hard-working/diligent messaging

Status quo messaging

Agreeable

Compassionate, caring, helping those in need messaging

Status quo messaging

Guess which side we started with to get to the test messaging?  Yes, you in the middle row, third from the right with your behavioral science sweatshirt.   The Donor side?  That’s correct.

The starting point for this test design was the donor,  or more specifically, our hypothesis that there are two donor types, differing by Personality.  This Personality difference tells how the messaging should change to give it the best chance of getting their attention and being acted on.

Sequence  and Interactions Matter

Donor first, “Object” (e.g. messaging, design, format) second.  The sequence matters.  Not because we want to label this test as donor-centric to satisfy a penchant but because the donor part is the most important.  It is the cause part, not effect. A then B.  Donor then object.

There are lots of other ways to expand on this,  and the level of analysis possible goes beyond the purview of this post. Suffice to say, we’ll approach this testing with more than one hypothesis and we’ll also seek a test design that factors in interactions.

Interactions are when two independent variables – Donor Personality and Donor Identity for example – combine as  super-powers and drive response up more than either factor separately: 2+2=5.

We’ll write more about these next level experiments in the weeks to come.

In our view, the sector is desperate (whether it knows it or not) for experiments and tests whose purpose is finding global insights rather than just a (random) test that beats the control.

Global insights on why people give to you are your superpower.  They can be applied across channels and communication type.  They are your ingredients to the 2+2=5 formula.

Kevin