An “A” for effort, a “C” for execution.  That’s how I’d grade a global survey in 63 countries and 59,440 participants testing 11  behavioral science interventions to impact people’s views and actions on climate change.

It’s a noble endeavor and they went beyond the usual WEIRD audience profile.  The middling execution grade is because buried in the report is this gem, “the results also indicate the impact of the interventions…depends on peoples’ pre-existing belief in climate change, supporting the claim that interventions need to be tailored to the characteristics of their audience [emphasis added]”.

That most experiments fail is known as is a key reason; the test idea affects people differently.  This should have been anticipated and baked into the execution cake rather than an after the fact explanation for why their interventions failed to move the needle – at all – on actual behavior change.  Having said that, the study lists 248(ish) academic co-contributors and the need for multiple Institutional Review Board approvals.  In that light, it’s a modern day miracle they pulled off a study at all.

Somewhere in the above is a how many academics does it take to change a light bulb joke.

So what did they test and find?  Each intervention is listed on the Y axis.  On the X are the outcomes they were trying to influence,

• Belief in Climate Change (vs. skeptic)
• Support for climate change policy
• Willingness to share info on climate change with social network
• Trees Planted.  This was an online task that directly funded tree planting, the more time put in, the more trees planted.

The vertical 0.0, dotted line is no effect, anything to the right is positive (though not necessarily statistically) and to the left, negative effect (ditto).  A few callouts,

• It’s relatively easy to sway opinion in the short-term.  Whether it has any staying power is the real question.
• It’s really hard to change behavior.  None of the 11 had any significant positive effect, most were negative and some significantly so.
• This study adds to the growing pile of evidence that media (outlets and social) is a growing cauldron of doom-scroll where negative is the food that feeds.
• Negative framing – i.e. climate change is really bad and bad shit is or will happen – runs a very high risk of getting lots of social shares in the doom scroll and the exact opposite of what you want for behavior change.

A case in point on different strokes for different folks is the wonky Binding Moral label.   It did the best in driving actual behavior (time spent on task to plant trees) and middling positive effects elsewhere.   This label is reference to Moral Foundation Theory, stating people make moral judgements along one of these dimensions.

1. 1. Harm/Care: I am motivated by empathy for others and value virtues of kindness, gentleness, and nurturance.
   2. Fairness/Reciprocity: I am motivated by concerns with justice, rights, and fairness in interactions.
   3. Ingroup/Loyalty: I am loyal to a group (e.g. social, geo) with elements of patriotism, self-sacrifice for the group
   4. Authority/Respect: I respect values of tradition and authority and virtues like respect and order
   5. Purity/Sanctity:  I am motivated by cleanliness, often tied to religious or spiritual beliefs valuing purity, sanctity.

Dimensions 3-5 tend to group together and it’s this grouping that is referenced as the uber Binding dimension.  Here’s the thing, we use this framework as part of our creative/strategic brief but which moral framing we invoke is inherently connected to the brand, the cause and the Personality of the donor we’re writing to.   Binding frames tend to work best with people high in Conscientiousness and as a poorer proxy, conservative people.

In fact, the details of the climate study findings show that Binding did work better with more Conservative people.  That’s great but because the interventions weren’t tailored in anyway the execution for Binding was limited and weak and there was no intervention for frames 1 and 2, which tend to skew towards people high in Openness and Agreeableness and as poor proxy, liberals.

What to take from all this?

• First off, it’s a cool study.  You can find a write up and a slick web tool to dynamically query the results.
• Negative framing should come with a caveat emptor rating.
• What “works” in social to get sharing – i.e. negative framing – may have zero value or worse, make people even less likely to give money/time.  And remember dear reader, more broadly engagement metrics have zero correlation with actual behavior.
• If your appeals are doom and gloom and fire and brimstone might it be that you are actually suppressing response?
• Stop thinking your test idea is so good it will work equally well on everyone in the random nth test group.  It won’t.
• Start thinking about a test that matches intervention/idea/message to the person receiving it and intentionally leaves other people out of the test group who are less likely to like it.  Or better, keep them in but state the two hypotheses up front – will work for Group A but not Group B – and break out results on back end to see.

Kevin