Amazon recently scrapped a machine-learning based recruiting tool.  Its sin?  It was discriminating against women.

Why would a machine pick up this very human bias?  The machine learned by looking at resumes submitted to Amazon over the previous decade.  Since the tech sector skews male, especially for technical roles (see chart from Reuters at right), the machine learned that male CVs were preferable.

While the AI was forbidden from taking gender into account, it got around this by:

• Penalizing resumes that included “women’s” (as in “women’s chess club captain)
• Dinging at least two all-female schools.
• Favoring verbs more commonly found on male engineers’ resumes like “executed” or “captured”

If it comes as a surprise that our myopia can be institutionalized, baking our biases into our black box algorithms, I strongly encourage Cathy O’Neil’s excellent book Weapons of Math Destruction.  It talks about how this type of discrimination enters our algorithms for loans, health insurance, criminal justice sentencing, voting, etc.

Readers have cautioned me against political statements.  We have a great diversity of thought in our rich nonprofit tapestry.  This is something I am working to take to heart; thank you to those who have pointed it out.

But I’m guessing I’m still on thick ice when I say this discrimination is bad.

Are we, at a clearly less insidious level, doing the same in our fundraising?  Are we turning away people who would want to be with us just because they don’t look like (whether that’s demographical or psychological) our current donors?

After all, when we advertise online, a common tactic (and one I recommend as effective) is to use a lookalike audience – taking our current donor base and finding people who look like those folks.  If your core donor identities are things like “parent” or “cat people” or “medical professional,” these services should give you a lookalike audience that looks very much like your current best audiences.

However, if your donor identity is “prostate cancer sufferers” or “parents who involve their kids in their philanthropy” or “crime victims”, Google and Facebook will give you people who they think look like those people.  But, since they are basing this on their searches and catalog purchases and whether they like Taylor Swift v Kayne, the results will be slightly more accurate than feeding the audience into a Ouiji board. The algorithm will do what it can, resorting to its baser instincts.  The same is true for offline modelers, except they will be looking at philanthropic records as well.

Moreover, if you have a database full of cat people and you run a lookalike audience/modeled list approach, it will give you an audience full of cat people.  Yay!  But it won’t tell you what would happen if you tried a dog appeal to dog people.  It can’t imagine that counterfactual.

This limits your options.  Imagine an organization that uses premiums all the time in both acquisition and cultivation.  Some of us probably know an organization like this.  It wants to test out of this or at least test into a two-track program where only the people who need premiums get them.

But the results say it can’t.  They test non-premium pieces to an acquisition audience and the anti-premium donors they acquire there don’t do well in the donor program, because it’s all premiums.  They test non-premium pieces to their donor audience and their pro-premium donors toss the appeal after failing to find their tchotchkes.  All because they have a well-worn path that they follow for all but this deviation.

Replace “premiums” in the above paragraph with “matches” or “multipliers” or “appeals to a specific donor identity” and it works just as well.  Replace “premiums” with “men” and you have Amazon’s former algorithm.  Say what you want about the challenges of nonprofits setting up data-driven marketing, but we can apparently set up self-reinforcing systems with the best of them.

It’s understandable to go to what we know and what our donors know.  But if you want new and different people to be a part your mission, it may require a different way of talking to those people.  It’s why we talk about different donor journeys for different donor identities, not just an A/B test.  When you see significant increases from the most minor of identity priming, you can see there’s a donor world out there hungry for a bit of differentiation, meeting people where they are.

Nick