But HOW do we measure advocacy success?

photo credit: Plastic Girl, via Flickr

As part of my continuing fascination with this question of how to evaluate ‘success’ in advocacy practice, I’ve been reviewing pretty much any documents I can find that purport to provide help for advocates and donors seeking to answer it.

The last one I read is this Annie E. Casey Foundation-funded guide to measuring advocacy and policy efforts. The guide has an appendix that includes examples for different evaluation protocols: measuring core outcome areas, evaluating strategic progress, assessing short-term incremental objectives, gauging organizational capacity, and documenting case studies.

The report makes some good points that must be addressed to advance this field:

  • Some ‘progress’ means holding the line against regressive proposals: how do we measure maintaining the status quo?
  • “Long-term” has different meanings for different stakeholders, so we need a common language of timeline expectations.
  • We shouldn’t oversell all advocacy as long-term societal change, when some efforts are really smaller-scale and more programmatic than revolutionary.
  • If we’re going to really change society, we need to frame all of our advocacy or policy change efforts as steps towards broader social change, but advocacy grantmaking is seldom seen as an investment in social transformation (so, then, it’s not surprising that it rarely yields that).

    But all of this discussion really got me thinking about the fundamental premise of this exercise in evaluating advocacy: that we need to determine what are the steps along the theory of change and measure how an effort is advancing along them. In this particular report, those indicators are defined as shifts in social norms, increases in organizational capacity, increases in alliances, strengthened base of support, improved policies, and, finally, changes in impact. And that’s when I started thinking: really, as an advocate for social justice (or, pretending for a moment, as a donor for the same), do I care if a particular group has improved capacity or more friends or stronger constituents or even a more sympathetic environment in which to advocate? I mean, I do, I guess, but only to the extent to which it yields what I really care about–the social change. If it doesn’t, for whatever reason, I’m going to be hard pressed to say that I think it was a real success.

    So, then, in pursuit of some way to legitimize advocacy and policy work, to bring it more in line with social service programming and its allegiance (rather belatedly) to evidence-based funding, are we reducing social change work to a caricature of its ideal self? Are we pretending that we know the formula for what makes a social movement happen and giving groups/organizations too much credit for going through those paces, regardless of whether, at the end of the day, they deliver the win? In so doing, are we emphasizing things that can be quite fleeting (a bigger coalition or better poll numbers, for example), or tangential (more staff or funding for the organization) and are unlikely to radically transform the lives of those impacted by the social problem in question? Should the only ‘interim’ step that counts be transformation in the lives of those affected, so that we can at least claim that as a victory if we fall short of the final goal?

    And if we abandon this quest for evaluation because we conclude that it doesn’t capture the radical nature of the work we want to support, then where are we left in terms of informing our strategies, convincing our donors, and elevating those with the greatest chance of success? Is the answer to do the same kind of evaluation but with a different articulated purpose? Or do we need entirely different kinds of evaluation, more retrospective in nature, that can then try to inform future efforts? And, particularly vexing, as we transform our work to make it more, well, transformational, by relinquishing the appearance of control and giving more power to the crowd, we make it that much harder to track our impact–how can we know exactly what difference we made, when we didn’t control (or sometimes even know about) everything that “we” did?

    As usual, I’m left with more questions than conclusions. What do you think? Have you evaluated your advocacy efforts? If so, what was worthwhile and what was frustrating about the experience? If not, what barriers kept you from that work? What questions do you still have, and what do you want to see explored as next steps in this exploration of advocacy evaluation?

    Advertisements
  • 6 responses to “But HOW do we measure advocacy success?

    1. Hi;
      We’ve got some common interests as you can see on my blog at drrickhoefer.wordpress.com and in my book Advocacy Practice for Social Justice.
      Anyway, to comment substantively, I use a version of a logic model to help me know what to measure and how much my efforts have advanced the cause. If you delineate ahead of time what you’re going for, it sure is easier to tell if you got at least part of the way there.

      Rick Hoefer

      • Thanks so much! I’ll check out your blog for sure. What types of courses is your book most commonly used in? And in your logic model, what are the primary ‘goalposts’ you measure? To me, the first challenge is to get advocates to believe that measuring advocacy success is indeed possible, and also worth their time, but then the really challenging work begins–what does success, short of substantial policy change, look like? Especially when the advocacy environment is so dynamic (so that, in some contexts, just not losing as badly can really be ‘winning’)? I’d love to hear any case studies you have for how measuring advocacy has made a difference, too, in terms of redirecting strategies and/or bringing about ultimate victory…that makes that first piece of persuasion easier! Thanks again for your comments.

    2. Melinda, you might be interested in the following article
      http://www.ssireview.org/articles/entry/the_elusive_craft_of_evaluating_advocacy
      by Teles and Schmitt. It touches on many issues you have cited.

      • Thanks, Jenn–I have read that, in the intervening years; there has been a lot of literature and discussion about advocacy evaluation, and it’s great to see a learning community develop around these questions. Thank you for reading!

    3. Pingback: Four Years in Retrospect | Classroom to Capitol

    4. Pingback: Anniversary Week: Your favorites! | Classroom to Capitol

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s