Challenges in Evaluating Advocacy

As I’ve discussed here before, one of the great challenges facing nonprofit organizations trying to integrate organizing and advocacy into their social service work (and, especially trying to get foundation or other outside funding to do that work) is in defining ‘success’ in the advocacy/organizing context and measuring the extent to which agency actions can be credited for that same success.

And this is a problem. It’s a problem because not all advocacy and organizing is very worthwhile, and the really effective work needs to rise to the top, just as in any activity in which nonprofit organizations engage. And it’s a problem because many donors use (in my opinion) the rather nebulous nature of outcome tracking in social change as an excuse not to fund it, which means fewer resources for this really vital work. And it’s a problem because we can’t maximally learn from what others are doing well (and not) if we don’t have common terms, common benchmarks, and a common mechanism for sharing and, then, building on, that collective knowledge.

So I’ve been doing a lot of reading about assessment in advocacy and organizing; I’ve talked with folks at some of the foundations and consulting firms around the country that are most advanced in this, and I’ve reflected on my own experiences as an advocate participating in evaluations. I have found a couple of resources that I think are really worth sharing, and I hope that they, and my reflections shared here, will be helpful to you as you set out to not only do social change work (yay! yay!) but also to do it intentionally well, to be strategic about how to assess it, and to then freely disseminate your results with would-be disciples.

When I was on the strategy committee of the Coalition for Comprehensive Immigration Reform, we participated in a pretty intense evaluation of our organizing and advocacy with Innovation Network. Recently, when I was on their website, I was totally blown away by the depth and breadth of resources that they have available for free. They are outstanding: online tools for setting benchmarks and conducting evaluations, a regular newsletter on evaluating advocacy, literature on the emerging field of evaluation, and more. It’s awesome, and all you have to do is register (for free). Check it out.

I also read through almost 80 pages of a pretty comprehensive report by the California Endowment (it’s good, but I don’t expect anyone else to want to wade through it–I did link to them below, in case you are interested). There are some good resources at the back of each report, though, so you might want to check those out–some online tools (many of which are also linked at innonet) and some literature. Here are my thoughts in reading through it, and thinking a lot over the past several days about this dilemma and how advocates and donors can work through it together.

  • Having a clear (and mutually-agreed-upon) theory of change is absolutely essential–we can’t bank on achieving the actual policy change that might be the ultimate goal, but if we know what needs to take place as interim steps towards that ultimate change, then we can count those accomplishments as outcomes, knowing that they are likely to contribute to our ultimate success. I can’t stress enough how much that kind of clicked for me this week. We need to spell out what needs to happen in terms of garnering support, changing public opinion, influencing the debate, etc…in order for policy change to occur–doing so will not only allow our investors to hold us accountable for those steps along the way but also make our own success much more likely. Some examples given in the report: shifts in critical mass, changing definitions, changing community or individual behavior, influencing institutional policy, holding the line. As we measure how well we’re doing on these goals, we’ll also be gauging how we’re advancing on our goal. Kind of a light bulb moment.
  • To really break through, foundations have to get over their overblown fears of lobbying. Otherwise, nonprofit organizations withhold some of the context of their work in order to make foundations feel safer, and then what they’re talking about is incomplete and sometimes almost nonsensical.
  • At the same time, though, we need an understanding of social change that far surpasses lobbying, or even policy change. We need to think about regulatory advocacy, legal advocacy, media advocacy, and community organizing as essential pieces of this work, just as important as legislative advocacy, depending on the target and campaign. Otherwise, we can confuse policy change (which is really just a means) with broader social change (the real goal). Doing so can lead us to prematurely declare victory or pursue an unnecessarily narrow range of activities.
  • We need capacity building. If an organization is an ‘advocacy organization’, then expecting them to turn on a dime and implement X social change campaign is reasonable. But when we’re talking about social service organizations learning an entirely new way to do their work, we need to invest so that they are then prepared to respond to opportunities (again, not just legislatively, but also in the community environment) as they develop. Along these same lines, we need to educate foundation Boards and Trustees about the long-term nature of social change and the need for investment beyond the 1-3 year term.
  • We can convert the process goals we commonly use in advocacy to the outcome indicators that foundations so want to see. It’s really just a matter of shifting our thinking. Instead of the number of meetings we held: the increase in the percentage exposed to the issue. Instead of the number of press releases: the number of times the organization was quoted. Instead of giving testimony: the organization’s statistics were used in a summary of the hearing.
  • We have to balance realistic and aspirational goals as we set our benchmarks. If we’re not striving, we’re not going to win, but if we are setting ourselves up for failure, we likely won’t be able to sustain the effort necessary for the long-term haul that is building social change.
  • Nonprofit organizations have to push back somewhat on the drive towards quantification of results; while these outcomes are important, they can also diminish the validity of alternative measures that may resonate more powerfully among the constituencies with whom you’re organizing, like storytelling.Organizers and advocates know that it’s not enough just to work really hard, or even to get a lot of people to show up or get a lot of attention (although those are great things!). We have to be making progress towards the kinds of social changes our society so desperately needs. We have to hold ourselves accountable not just because we can get more money that way, or because it makes us look good, but because the marginalized communities with which we’re working have been sold inferior goods and services for far too long–they deserve to work with people who can and will deliver. We need to learn what we need to measure, and then measure it, and then not be afraid to shout that it works!

    Challenges in Evaluating Advocacy Part I

    Part II

  • Leave a Reply

    Fill in your details below or click an icon to log in: Logo

    You are commenting using your account. Log Out /  Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out /  Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out /  Change )

    Connecting to %s