Fundraisers … Beware Of Vanity Metrics
If you truly believe that fundraising is mostly art’, not ‘science’, then stop reading. Chances are you’re going to neither understand nor care which metrics matter in fundraising.
It’s fashionable these days for many nonprofit fundraisers and their consultants to claim they’re ‘data-driven. Problem is, most focus on the ‘data’ part of the slogan, but few understand the requirements of the second word — ‘driven’.
In a fascinating new book, Lean Analytics, authors Alistair Croll and Benjamin Yoskovitz argue: “If you have a piece of data on which you cannot act, it’s a vanity metric. If all it does is stroke your ego, it won’t help. If you’re aiming for success you need your data and metrics to inform, to guide … to help you decide on a course of action.”
Although Lean Analytics was written mostly to guide entrepreneurs on start up businesses, it got me thinking about how many of the metrics commonly used in our sector are really little more than what the authors call ‘vanity metrics’ — good for the ego, fodder for the next board meeting or next conference, but really not useful for action leading to improvement of the bottom line.
If the metric, and the data behind it, doesn’t trigger the question — “What will I do differently based on this information?” — you probably should not be spending time or money collecting and analyzing it.
In essence, part of the problem with ‘vanity metrics’ is that nobody does anything with them. But more importantly, and dangerously, they are often used to drive absolutely bad decisions. For example: “If I put more spend into online search advertising, it’s a quick, guaranteed way to drive up the number of website visitors.” A meaningless strategy. A meaningless result!
Here are my picks for ‘vanity metrics’ of which fundraisers should be beware, both in the online and offline realms:
Benchmarking. In my ‘nobody does anything with the data’ category, the number one ‘vanity metric’ is benchmarking. While I applaud Chuck Longfield and Blackbaud’s Target Analytics for creating and propagating the concept, my problem is that individual organizations seldom use it to question and guide new actions. Instead, if they’re ahead of the benchmark in their sector, they use that stat as proud justification for doing nothing except boasting to their board or CEO. If ever a brilliant metric concept were mis-applied for the sake of vanity, it’s this.
Number of Hits. A fossil from the early days of the web. If you have a big site with many elements there will be lots and lots of hits. So what? Better to count the actual number of people involved.
Number of Page Views. Just a tiny bit better than ‘hits’. Unless you’ve discovered how to raise money on page views, this is a metric without actionable value.
Number of Visits or Number of Unique Visitors. Sheds no light on why they visited, why they kept sticking around or why they left. Pretty much worthless.
Number of ‘Likes’, ‘Friends’, ‘Followers’. Simply a popularity contest. Only if you know how many followers will take action when you make a request do you really have an actionable metric. Most organizations don’t use it this way.
Number of Email Addresses on File. The issue isn’t size of the list, although it may impress the CEO or board; the valuable and actionable metric is how many folks will open their emails AND take action.
Acquisition Response Rates. Amazing the number of direct response fundraisers who measure ‘success’ or ‘failure’ by response rates, as in “Our rival gets only a .65% rate and we get 1.2%”! Pretty much worthless because it tells you only how a specific campaign went, and raises almost no “what should I do differently” questions, other than “what should I do differently to get the response rate up?”
I certainly realize that some readers are growing angry over these remarks. But I find the whole concept of ‘acquisition metrics’ comical, reflective of a singular obsession with NOTHING BUT campaign metrics and various manipulations that produce only variations of the same numbers. Columns and columns of redundant, non-useful metrics — created by just switching around the numerator and denominator on response rate, average gift, average cost, total cost and that magic output … ‘cost to raise a dollar’.
Why not a set of useful metrics that could be done on the same report that would be forward looking and remind the report reader of what the goal truly is?
Stay tuned. These are the metrics that count, which I’ll discuss tomorrow.
Roger
7 responses to “Fundraisers … Beware Of Vanity Metrics”
Behavioral Science Q & A
Integrating an individual giving appeal with other communications from a charity can have both positive and negative effects, and the outcome largely depends on how it’s executed. Advantages of Integration Brand Consistency: Maintaining a consistent appearance and messaging across all communications can reinforce the org’s brand identity and strengthen brand recognition and trust among your […]
Read Full Answer
I’m not aware of any in-market tests specifically comparing recurring vs. gift frequency language. I suspect the answer might not be the same with all gift frequencies, nor with all people. It sounds like a great opportunity for you to test and find out what works for your audience. Based on the literature, here’s a couple […]
Read Full Answer
Based on what we know from existing data, those renewal notices can actually be pretty effective in getting people to donate. They tap into our psychology – creating a sense of urgency, reminding us of past support, and using personalization to make the message hit home. They’re playing on our natural tendencies to feel obligated […]
Read Full Answer
Interesting question. I had a quick look at the testing done on this topic. On the positive side, in all cases, over half of donors decide to cover the fee. In some cases, it goes as high as 65%. Not a negligible percentage at all. Here’s another test from iRaiser showing consistent results (see point […]
Read Full Answer
There’s just one thing to consider when designing a supporter journey: the supporter. More specifically, you need to take into account: Who the supporter is i.e. their identity, which is the reason they support this cause, and their personality, which describes the way they “see” and process the world. These will determine the kind of […]
Read Full Answer
I’m not an expert in this but a quick search surfaced this article on the effect of tax reforms on 2019’s charitable giving. The researchers didn’t find a reduction. Actually, they observed an “increase in charitable contributions in 2019, even with the lower tax rates and the dramatically smaller number of taxpayers who itemize their […]
Read Full Answer
Thank you finding words to describe a pervasive problem.
Whenever I get together with fundraisers these days, conversations almost always turn to “the system” as the malady behind symptoms like “vanity words.” Many organizations have impossibly high expectations (often from board or top-level mismanagement or misunderstanding of the business) when they hire new development personnel. Often, those who hire in are either fleeing another impossible fundraising situation or are using the move to increase pay or title because their prospects are limited at their former employer. To get the job, new hires over-promise, knowing they will have about three years to get a few impressive “wins” before the honeymoon ends and it’s time to move on to the next job. This constant turnover makes it easy for boards and executives to continue functioning in a “reactive loop” that propagates the system. Job interviews, resumes, board reports, etc., become virtual “vanity word porn” as everyone–board, executives, and fundraisers–work in a sick business environment. Do I overstate this?
Yippee. You agitators keep agitating. I count on you to reveal the emperor (or empress) without clothes. What are we measuring and why? What value is the metric? To what degree does the metric make us think and explore and then take action?
I’m hoping we measure/examine process, not just results. I’m hoping that we measure relationship building, not just solicitation.
I’ve collected quotes forever. Quotes from novels, not just business books. Here’s a favorite of mine: “Data itself is nothing unless one uses it as a resource from which to draw conclusions.” In the sci fi book Dune: The Machine Crusade by Brian Herbert and Kevin Anderson.
I look forward to the next Agitator for ideas about metrics.
To add to the vanity list: reporting impressions and/or clicks as the end measure for Google ad words instead of carrying it through to the action goal (aka conversion … although that word already has too many layers of meaning in the fundraising world!”)
Leave it to the Agitator to pull the wool up and away from our eyes. But as a realistic optimist, I fear it’s going to take more than this sterling post to stop these inane practices. Regardless, I will now find myself straining to avoid asking potential clients if they mean they have ‘ego-driven’ programs when they tell me they have data-driven programs.
Sometimes I feel like all this internet stuff is a brave new world that we are inventing as we go. More often I feel like I am shooting from the hip in the dark, guess about what will work online.
In this environment, we are grappling for something, something that will give us an indication of whether we are being successful, or even if we are heading in the right direction.
It’s probably natural for novices (and naive board members) to latch on to these metrics, because they don’t know any better. I look forward to your suggestions tomorrow!
Great reminder – if you can’t do something with a stat, it’s not worth collecting. But I personally wouldn’t toss “benchmarking” out without first asking how (if) you can use it. In the past, working without a lot of support from management (as in no annual reviews and goals, or ones that were essentially meaningless – and nothing much the rest of the year, either), I used benchmarking to push myself to do better. If my colleagues in the same vertical are doing X and I’m only doing X – 10%, what can I do to be as good as – or better! – than the majority? Benchmarking can serve as the goals that you aren’t getting elsewhere (other than the oh-so-helpful “keep it up” or “do better!”). Benchmarking can give us measurable goals to strive for when wisely applied.
Pam,
I’m not suggesting ‘tossing’ benchmarking. Far from it. Benchmarking can be a valuable metric when used for other than sheer ego or vanity purposes. As you note, using benchmarks to trigger inquiry into what others with higher benchmarks are doing –exactly the process you describe in your Comment.
Roger