California banned thin plastic grocery bags. The number of those bags fell by roughly 40 million pounds a year. Look only at the checkout counter and the intervention worked beautifully.

Shoppers, however, had been using some of those bags again. They lined small trash cans, handled pet waste, and performed the various grubby jobs for which nobody reaches for a tasteful canvas tote.

When the free bags disappeared, people bought substitutes. Sales of small trash bags jumped 120%, medium bags rose 64%. Those replacement purchases added about 12 million pounds of plastic, wiping out roughly 30% of the apparent reduction.

Economists call this leakage: reduce consumption in the place being regulated and some of it reappears somewhere outside the regulation.   The “win” depends on how tightly you narrow the lens.

Direct response can get lured into the behavior narrowing trick.  See a weak number and turn it into the definition of the problem: response rate, cost per donor, form completion, event attendance. Then we change the offer until the number behaves.

If acquisition response falls we can add a tote, blanket, calendar or stuffed animal and we may see response rise.

Yet the package may have recruited people who valued the item more than the cause. The missing commitment then reappears as weaker renewal, lower net value, added fulfillment costs and a donor file containing people waiting for the next piece of branded household clutter.

That is fundraising leakage: an apparent gain in one place that creates or reveals a cost somewhere outside the metric, department or time period being watched.

Before calling an intervention successful, ask four questions:

1. What behavior actually changed?
2. What did people do instead?
3. Which costs or consequences sit outside our measurement window?
4. What outcome matters across the whole system?

This is where behavioral science earns its keep. Its first contribution comes before the tactic, forcing us to define the behavior, understand the function it currently serves, anticipate substitutes and decide how broadly success should be measured.

The next time a metric improves, delay the victory lap long enough to ask where the missing problem went.

It may be sitting in another report, wearing a thicker bag.

Kevin