Is Your Testing Divorced from Reality?

June 30, 2021      Kevin Schulman, Founder, DonorVoice and DVCanvass

“If only we could get more people to give and do so more regularly.”

” Let’s devise a test. ”

The HiPPO { Highest Paid Person’s Opinion; the one usually in charge) in the room says,  “More people will give if we change the landing page to have more/less…text/pictures/fields.”

“Great, we’ll do that test”,  says the non HiPPO in the room.  This person, who’s also an Agitator reader,  and especially keen to learn more about Personality in fundraising, quietly but confidently adds these points,

  • the default for people will be to do the same thing they did before – not give for the non-givers and give the same amount for the givers.
  • And the donor’s default setting will be determined, in large part, by their personality.
  • And since personalities are different, defaults are different
  • So, we’ll want to have a specific test idea aimed at a specific Personality type to have the best chance of success.

ZZzzzzz….ZZZZzzzzzz.   Mrs. HiPPO, are you asleep?   “Argh, sorry, I must have dozed off.  My goal is to keep things simple.  And if need be, simpler than possible.  Harrumph.”

The enterprising, intellectually curious staff person realized the KISS (Keep It Simple, Stupid) method in this charity was overly focused on the second “S”.  But, he’s got a Conscientiousness personality and felt duty bound to share the research he’d done with a behavioral science agency (rhymes with TonerChoice).  That research identified the Personality profile of their donor base and provided details on how to target a messaging to each group.

And so, while simultaneously updating his resume, he shared this matrix highlighting the organization’s three most prevalent, Personality defined, donor segments.  He noted these three groups need different messaging, imagery and even form field names.

The room fell silent…because everyone had left about five minutes ago.

Click to Enlarge

While this parody is also divorced (slightly) from reality it still lands dead-on in how 99% of our testing plans assume everyone is the same.  We ended a post last week with a soft plea that the A/B, test with its random nth assignment of humans into one of the two conditions die a quick death.  We also lamented that’s unlikely to happen. So maybe this post is just more windmill tilting.

HOWEVER…. we tilt because we care.  We care that our sector wastes an enormous amount of time and money on testing to nowhere, with test ideas divorced from why people do what they do and methodology that mistakes the lead standard for the gold standard by using the random nth to split people into test and control groups.   In so doing, we violate a most basic maxim of humanity:  we’re different.

Kevin