There are some phenomena that are impossible to measure without effecting some  change.  Think of your tire pressure gauge – to measure the pressure in your tire, you have to let some air out, thus changing your tire’s pressure.  This is so ingrained in our lives and our physics that there are times that subatomic particles can interfere with each other only if no one is watching (here’s the study if you are a physics geek).

If our universe is set up this way, it stands to reason we humans are not immune.  And we aren’t.  We even behave differently when fake eyes are watching us – increasing our donations in two different studies, for example.

As for surveys, in the 2018 AP Votecast, voters were randomly assigned to be asked about their sexual orientation (or not).  Among those not asked about their orientation, 46% of men and 40% of women said they own a gun.  Among those asked about their orientation, 50% of men and 42% of women said they own a gun.  Asking about sexual orientation cause some people to either remember or make up the presence of  a gun in their home.

Think what you would be willing to spend to get the response rate on a mail piece to go from 4.6% to 5%.  That’s the level of impact asking about sexual orientation had on purported gun ownership among men.  Smarter people than I can sort through the psychosexual implications of the question and of gun ownership and all that.  I’m interested in the simple phenomenon: the act of observing changed the observed.

Now that you know you have this power, how will you use it?

First, hopefully, by asking constituents’ opinions.  The mere act of asking someone’s opinion, whether you do anything with it or not, increases their liking for you.  A study from the for-profit world found that customer retention increased 15 percentage points between those people who had a complaint but didn’t tell anyone (31% retention) and those who made a complaint that wasn’t resolved (46%).  In short, you can increase your donor retention substantially simply  by asking people their opinions and then doing nothing about it.

Of course, that’s not the recommended strategy – it’s clearly better to do something about the information that people give you than see if you can get it through the novelty basketball hoop above your trash can.  We play the long game.  Eventually someone will figure out their opinion doesn’t matter in that world.  But they will figure it out faster if you don’t ask at all.

Once you decide to ask people you have to decide what the goal of your asking is.  Is it to gather information?  Is it to push people to an outcome?  Or it is some of both?  On one end of the spectrum, you have traditional polling that works to eliminate biases when possible.  On the other side, you have push polling and surveys with questions designed to get ” yes ” responses (“1. Do you believe in America?”  “2. How about puppies?”) with a donation ask at the end when someone is theoretically full of “yes”.  Right now, one of the control ads for the U.S. presidential re-election campaign takes a person to a Mainstream Media Accountability Survey with questions like “On which issues does the mainstream media do the worst job of representing President Trump? (Select as many that apply.)”.  (If you don’t click on such ads, I highly recommend you do so: partly to see the social psychology going on and partly because they are called cost-per-click ads for a reason.)

Mostly likely, you are going to be asking your donors for their unbiased thoughts about themselves and their experiences with you.  But thanks to the mere-measurement effect, asking someone about their intent to donate again makes them more likely to donate again.  I would recommend asking this question, then, even if your goal is pure knowledge – might as well get an additional benefit.

Still, you need to take care to avoid unduly biasing your audience.  I used to love asking the question “If you could change one thing about X, what would it be?”  It turns out that the easier it is to recall something, the more you think it is true.  Thus, people who are asked “one thing” questions are swayed in the direction of the question, whereas people told to list as many things as they can are swayed away from the question.  If I wanted to make people feel satisfied with an experience, then, I should be asking what’s the one thing they like best and what are the 17 things we can be doing better.

All of this is to say that the first step is getting surveys going, listening to the responses, and acting on them.  A survey design professional can be a huge help to either mitigate these types of effects of observing on your donors or to aim them in a positive direction.

Nick