Behavioral economics (aka “nudges”) is the observation that sometimes people behave irrationally.   You can google it and find long lists, infographics and diagrams listing reams and reams of biases.  It seems all the cool kids have a bias they invented.

You can also find lots of non-scientists and non-researchers espousing expertise because they read a popular business book or two – e.g. Nudge by Thaler and Sustein, Thinking, Fast and Slow by Kahneman.

There are two problems here,

• 1. the non-scientists have a thimble full of knowledge they mistake for an ocean.  Would you consult a pre-med undergrad with a medical question?  Why do the equivalent it when spending donor dollars and meaningfully trying to impact society?
  2. You can ignore pt. 1 entirely if you like because pt. 2 is the death knell for the field as currently practiced by scientist and poser alike. Nudges don’t work (as currently implemented).

Let’s dig in on pt. 2. Many social science disciplines are experiencing a replication problem, meaning folks can’t do the same experiment at two different times and get the same answer.  That ain’t good.   Nudging is no exception.

A meta-analysis of a huge sample of peer reviewed, published studies on nudging that took into account the severe effect of publication bias (i.e. nobody publishes tests that don’t work) found zero evidence of nudges working.  This is a very wonky, sophisticated analysis but all those black squares and error bar lines that sit on or include the dotted line and to the left of it ain’t good.

But, this study buries the headline and incorrectly pronounces the nudging patient dead.  Here is the real headline,

“All intervention categories and domains apart from “finance” show evidence for heterogeneity, which implies that some nudges might be effective, even when there is evidence against the mean effect”

The plain English version: nudges work on some people some of the time.  But if you run the standard A/B test and divide people randomly into control and test then you’ll see no effect.

Heterogeneity means different outcomes for different contexts and different people.  If you don’t understand human motivation and the role of Identity, Personality and context in decision making then you’ll never, ever have repeated, sustained success with nudges or any other test idea for that matter.

Factoring in the head-nodding-in-agreement notion that people are different and that contexts are different doesn’t guarantee test success.  Hell, far from it.  But it does guarantee test learnings – see the charity: water case study here.

Kevin