The Great Data Debate

“There can be no turning back on data. We’ve got to accept that measurement is here to stay, and make it work”. So said Geoffrey Canada, the driving force of the Harlem Children’s Zone, on June 2nd at the annual conference of the Committee Encouraging Corporate Philanthropy.

Mr Canada had discovered the importance of measurement in the non-profit world as he grew HCZ – which focuses on helping children in poor areas succeed, especially through education – into one of the most admired newish non-profits in America, where it is being scaled up, by replication, by the Obama administration through its Promise Neighborhoods programme. Apparently, at one point some of his colleagues at HCZ felt that Mr Canada had become so obsessed with measurement that they started saying “Geoffrey cares more about data than children”, to which he replied “if you really care about children, you have to care about data.”

This captures in a nutshell a crucial debate that is intensifying across the social innovation sector. Our arch critic, Michael Edwards, loves to bemoan the evils of using measurement in the non-profit world, and usually gets plenty of sympathetic nods from non-profit workers in the audience when he does so.

We are certainly not uncritical advocates of measurement. As economists, we are all too aware of the famous dismissal of economists as people who “know the price of everything and the value of nothing” – which is another way of saying you can measure the wrong thing and also, perhaps, that the easiest things to measure (such as a price) may not be the right things to measure. We also recognise that there is a temptation, often succumbed to even by philanthrocapitalists, to deploy data in a defensive way, to cover their back in case things don’t work out rather than to improve the chances of having the best possible impact. We are hugely suspicious of people who focus on data to the exclusion of other less numerical information, including gut instinct.

Yet, in the end, Mr Canada is right. In a world of scarce resources, not to try to measure how well those resources are used is, ultimately, a failure to care enough. The challenge we all need to wrestle with is how to get the measurement right, so it really helps us to achieve the change we want.

This seems to be a task to which many in the corporate giving world are warming, as blogger Alice Korngold reports. So, too, the Obama administration, as is evident from this blog by Peter Orszag, the director of the White House Office of Management and Budget. On the same panel as Mr Canada, Patrick Corvington, the CEO of the US government’s Corporation for National and Community Service, indicated how radical this move towards meaningful impact metrics could be. “No longer will we equate success with increasing the number of Americans who volunteer,” he said, promising to unveil soon a raft of new measures designed to indicate whether the service his organisation encourages is making the difference it is supposed to.

These new measures should certainly be scrutinised carefully, to make sure they are really shedding light on the right things and do not inadvertently create incentives for undesirable activity. But that scrutiny should be undertaken with a view to improving the data, not, as we fear will be the case for Mr Edwards, to provide an opportunity to say “I told you so”. The money available for building a better world is too scarce, and the problems too pressing, for critics of data to ever indulge in Schadenfreude.