Is “Which Test Won?” the Right Question?

May 15, 2024      Kevin Schulman, Founder, DonorVoice and DVCanvass

Charity X believes in in the importance of a healthy home where families can live together. They provide rehabilitation services to working families who are trying to develop themselves to follow American traditions and support their communities.

Charity X believe every person deserves the protection of a home. They provide free critical repairs to the homes of low-income Americans as well as financial support to protect low income families who are susceptible to home loss. This support works to ensure every individual has the right to a home.

  1. Which description of Charity X do you personally prefer? 
  2. Which description wins as part of a head to head test?  

The first question is answerable and no right or wrong answer.  But is the second question the right one to ask?  In 99% of testing, it’s the only question.

Here’s the rub, both descriptions are accurate as both are based on information on the charity’s website.  Same brand, same mission, very different framing.

One resonates with you more but there are people not like you who prefer the other version.  In a typical, random nth test the winning framing comes down to whether the list is mostly people like you or not like you.  That’s hardly optimizing for success.

Instead, the hypothesis behind this test was that we have different people with different values who will equally support Charity X if we match the moral framing to their respective values.

The first descriptor emphasizing tradition and working families was written for more Conservative and Conscientious people.  The second emphasizing protection and the right to a home was written for more Liberal and Agreeable people.

There isn’t a single winner because they were never intended to be head-to-head.  We have two audiences, not one.  Each deserves messaging that abides by the brand while matching the values of the recipient.  It’s very easy to find conservative and liberal people.  A better option is going one level deeper, to root cause and using Personality Tags.  Conservatives tend to skew Conscientious, Liberals tends to skew Agreeable and Open and you’d want a very different framing for the Open Liberals vs. the Agreeable Liberals.

Greater connection should be a part of your growth plan.  And either everyone the same on your donor file or they aren’t.  Either way, it’s a problem and asking which test won won’t fix it.

Kevin