Hidden in many “losing” test results is a test idea that worked for some people and not others.

Here are experiment results for World Vision among prospective donors randomly split into the control with altruism messaging and a test with self-interest messaging.

Nothing to see here.  The test lost. Discard, move on…unless you designed your test assuming people will respond differently based on who they are as people.

The World Vision test was such a test.  The bar chart is what we’d expect, no difference, if you mash everybody together and random nth your way to ignoring the differences between people.

Here’s the real test.

• Control:  The altruism appeal, which we dubbed the control because it’s more typical in the sector – had the ask framed as,
  • “Any donation you make will improve the happiness and wellbeing of an African family.”

• Test:  The self-interest appeal had the ask framed as,
  • “Research by psychologists shows that donating money to charity increases the happiness and wellbeing of the giver.”

The hypothesis was not that the test would beat the control  The hypothesis was,

• The Control message will work best for those whose personality is hard wired to be high in Agreeableness.  These people, more than others, are motivated to provide care and help others.
• The Test message will work for those low in Agreeableness and higher in the innate Neuroticism personality trait.

Why Personality?  It travels well.  It’s part of you.  It is measurable, targetable, we know how to message to traits and most importantly, our personality determines much of what we pay attention to and act on.

Framing appeals to match who I am versus who I am not seems obvious except when obvious is in short supply.

The experiment included measuring Personality traits after the donation in order to break out donation behavior by trait and by control/test.  As predicted, the test won among those low in Agreeableness and higher in Neuroticism and lost for those high in Agreeableness.  You can’t produce this bar chart unless you group people purposefully; random nth is the opposite of this.

Framing matters.  But it matters differently for different people.  You do have different donor segments but unless you’re digging deeper into their why, those segments are hidden from you, mashed up in blue and green bar charts and false negatives.

Kevin