Every new, innovative idea that walks into a nonprofit gets frisked at the door. The old ideas, practices, and ‘controls’ stroll right past security. Nobody asks them for ID.

Barbara O’Reilly put her finger on the symptom in her latest Gut Check at Windmill Hill Consulting: “Why does everyone want constant innovation, but no one wants to take risks?” Her answer is fair and true as far as it goes. Budgets are real, boards ask questions, the familiar feels safe. And she’s right that new ideas and innovations deserve a budget and room to fail.

But fear of risk isn’t the disease. An Agitator reader asked me the same question more than a decade ago, and my first answer was the easy one: risk-averse, under-resourced, short on imagination.  But, I never hit “Send” on that answer because the real answer is uglier. And more fixable.

We Demand Perfection From the New and Never Audit the Old

All too many nonprofits and their consultants refuse to try the new thing unless it works 100% of the time. And they make that demand without the foggiest idea how often the old thing fails.

They reject a predictive model because it’s right 80% of the time — never asking how often their RFM segmentation is right. Maybe 60%, maybe 50%. Nobody knows, because nobody checks. We dismiss the phone for sustainer acquisition because it “costs more” and “bothers people at dinner,” without bothering to calculate that mail fails at acquiring monthly donors about five times as often as the phone does.

Imagine refusing to replace a leaking roof because the roofer won’t guarantee the new one will never leak. Meanwhile, you’ve never counted how often you drag out the buckets, what the water damage costs or how much you spend patching the same holes. You scrutinize every possible failure of the replacement while treating the old roof’s failures as routine maintenance, as if emptying buckets is now part of the business model.

Five Out of Six Tests Lose. We Call That “Proven.”

Direct mail acquisition testing is the purest example. Fewer than one in six new packages ever beats the control. That’s an 84% failure rate, earned slowly and expensively: the ink dries, the post office delivers, returns dribble in over weeks and months, and somebody finally analyzes them. Most of what we test, the late Ed Mayer called “whispers” — envelope colors, fonts, the shape of the reply form. Incrementalism to nowhere.

Yet when someone brings multivariate testing, conjoint analysis or predictive modeling into the room, the trade dismisses it as witchcraft — newt’s eyes and lizard tails — because it can’t promise a sure thing.

A real case from the hands-on practice over at DonorVoice.  A very large mailer — tens of millions acquisition pieces a year — wanted to beat its control with eight new response forms from various creative shops. A conventional wet test with statistical confidence meant mailing 320,000 prospects, spending $92,000 and waiting at least 12 weeks. Instead, they ran a multivariate pretest online for $25,000. It evaluated not just their eight versions but all 512 combinations of the components. It identified the winner in under ten days.

None of the eight forms the client and its consultants picked came close.  A multi-million-piece program and the client had no idea what its failure rate was costing it.

The Mailbox Was Always People-Based. Digital Never Was.

Kevin made the same point to me earlier this week from the other end of the channel mix — with considerably more profanity than I’m going to print.

Mail has one enormous, underappreciated virtue: you know exactly who you mailed. Same with the phone. Digital — the channel everyone wants to be “first” in — is the opposite. You set some loose targeting parameters, hand the machine a goal (impressions, cost-per-click) and let the platform optimize. Who actually saw your ad? You have no idea. The audience stays forever hidden.

Kevin’s team over at DonorVoice built an alternative to help clear the digital fog. Start with a list of sourced, new names, curated for fit with the mission.  New means really new, not the co-op names or list rentals you’ve exposed countless times over.  This is new to you and you to them sourcing but they have what he calls “Identity/Mission” fit.  Third party data, as messy as it is, does have signal.  How people show up in the data partly tells you who they are.

The question shifts from “how do I grow?” to “what brand/fundraising media mix converts in-market people to donors?”   One example, split your list in thirds. One cell gets get click-to-donate conversion ads, another gets brand ads plus conversion ads and the third gets all that plus a mailing. Because it’s people-based, you know what exactly what you spent on each cell and the bank account tells you exactly what they gave. No cookies and no attribution fairy tales.

“I’m looking at Roger Craver’s name,” Kevin explained. “I know I spent X against him. Did he donate or not?” Now you can see whether the mail earned its keep, and whether brand advertising brought more people in the door even when the upfront ROI looked thinner.

When I asked him how other fundraisers react when they hear about this innovative process.  “They hear the pitch and say yes.” he noted, “But then the questions start. When exactly do we break even and can we get all the results at year end?”

The Burden of Proof Sits on the Wrong Party

Those sound like smart questions but they load the entire burden of proof onto the new thing, while the status quo enjoys what Kevin calls “a blissful acceptance: nobody checks how often it fails, and familiarity passes for evidence.”

So here’s my answer to Barbara’s questioner. Fundraisers don’t hate risk,  they take enormous risks every year — mailing millions of pieces that lose five times out of six, pouring budget into digital black boxes they can’t audit. What they hate is visible risk. The familiar failure stays invisible because nobody measures it. The new thing’s failure would land on a report with somebody’s name on it.

Before you ask a new idea for a guarantee, ask the old one for its numbers. What share of your acquisition tests beat the control? How often is your segmentation actually right? Which of last year’s fourth-quarter dollars would have come in without the digital spend or the SMS you layered on top? If you can’t answer, you don’t have a proven program. You have a habit.

And please do the math. A new approach that beats the control half the time crushes a “proven” practice that beats it one time in six. Every time.

You don’t have to love failure. You do have to count it — your old ‘control’, status quo  ideas, not just the new kid’s.The new idea doesn’t need a guarantee and the old one needs an audit.

Roger