The Stanford Social Innovation Review had a special series on measuring social impact this spring, full of so many terrific insights that it took me quite awhile to sift through all of the articles and, then, compose my thoughts at least somewhat, to post here.
I’d love to discuss any of the pieces, and I welcome your responses to my reactions, too.
Above all, I’m very glad to see this conversation within this sphere; if we’re not asking what our true impact is, we’re missing the only metric that really matters:
Are we making the difference we intend, and that so desperately needs to be made?
- It is somewhat disturbing, really, that an article entitled, “Listening to Those who Matter Most, the Beneficiaries” even still needs to be written. The article highlights some promising beneficiary feedback initiatives around the world, giving detailed descriptions of how the perspectives of students in struggling schools and of patients in health care settings are being used to inform program innovations. It is my hope that the challenges outlined and the case made for the advantages that accrue when participants (I like this term better than ‘beneficiaries’) actively shape activities can both help to push public policy in this direction, too. Then we can really get to impact.
- There is a brief outline of a larger academic paper centering on how to evaluate the effectiveness of civic engagement and advocacy efforts. Importantly, it incorporates multiple stakeholder perspectives, but I am still dissatisfied; it feels, to me, too much like asking about participant ‘satisfaction’, which may or may not be a good proxy for efficacy, even in the context of civic engagement (which, after all, is designed to foster feelings of good will within the community).
- My advocacy evaluation work focuses on using evaluation to improve performance, but we are often constrained by the inadequacies of our evaluation approaches to capture the rather elusive nature of advocacy and social change activities. This dynamic, between measuring to improve and improving measurement, is the subject of one of the articles. It mostly summarizes a workshop session related to evaluation, but I appreciate the inclusion of several specific and innovative approaches. Sometimes we have to get a bit ‘meta’, stepping back from our work in order to invest in the capacity to perform it better.
The folks at SSIR have been leading the field on the question of how to really define ‘impact’, and so it’s not their oversight, but I do think that we, collectively, need to spend more time within our organizations, our profession, and our field really clarifying what impact means, and what it looks like, in order to ensure that we will, indeed, know it when we see it.
But maybe approaching it from this direction–how can we measure it, before we are necessarily sure what it is, should offer some appeal.
If one of the reasons we have excused ourselves from getting serious about setting the bar for ‘impact’ accurately has been that we don’t know how we will be able to know when we’ve reached it, then perhaps addressing the latter will light a fire under us for the former.
A better measure for a better system
How should we measure ‘well-being’?
One of my intellectual interests relates to how evaluation and social indicators can focus our collective attention on the problems that need to be addressed, setting better benchmarks toward which we should aspire.
And one of my great passions is around reducing political, economic, and social inequality, to build toward a more just future.
And, here, these two worlds align.
Because we need some better measures of how we’re doing.
I don’t mean the U.S. poverty line, although clearly that needs to be revamped.
But, here, I’m thinking more of the underlying issue, not poverty but what creates the conditions for it.
We need a better measure than Gross Domestic Product per capita, because, clearly, an increase in GDP doesn’t always translate to an increase in well-being.
Look at how much more we spend on incarceration today, which is tied to an increase in GDP, when it’s clear that people aren’t benefiting from that particular outlay.
We have the Gini coefficient, which measures inequality, although, perhaps not surprisingly, it doesn’t hold much cachet with policymakers or even pundits in the U.S.
Something like the 20/20 ratio, which compares how well the bottom 20% are doing, compared to the top 20%, would be even more helpful, I think.
Or the Hoover index, which calculates how much redistribution would be needed to achieve total equality.
I’m certainly no economist–or mathematician–but an indicator that could clearly indicate a person’s likelihood of leaving poverty, or leaving the bottom 20% or so, could, if inserted into our understanding about our economic system, help to crack the myth of ‘rags to riches’.
So why do we use GDP per capita, when it so clearly fails to capture so much of what we really need to know, and distorts so much of the picture?
There are better measures out there, and we certainly have the technical capacity to shift to them, or even to develop something else, if we really wanted.
I can only conclude that our stubborn clinging to something woefully inadequate has much to do with how we come out looking relatively good according to that measure, and pretty blatantly unequal according to others.
If we’re not winning, after all, we can always move the goalposts.
But I think that, while metrics are surely not everything, having better measures would really help.
You manage to what you measure, after all, and, if we had some consensus about what we were working toward, we’d at least have a shot at getting there.
Share this:
Like this:
Leave a comment
Posted in Analysis and Commentary
Tagged evaluation, inequality, poverty, social indicators