Measurement Fetishism

Published on Thursday, 16 July 2015
Measurement Fetishism

 

At my first university job, in the early 1970s, a professor whose expertise was statistics and research methods said, “If you can’t measure it, it doesn’t exist.”  I thought, then and now, that that statement is both extreme and absurd.  More recently, in a World Bank blog, Harry Patrinos quoted management guru Peter Drucker as saying, “If you can’t measure it, you can’t manage it.”  While I am afraid that, to many, this statement sounds more reasonable than the first, I think it just as extreme and absurd.  Unfortunately, we live in a world of measurement fetishism, much to our detriment.

To begin with, you can manage things you don’t measure.  You have to.  Everyone does.  Patrinos tries to make the opposite sound like both common sense and proven by research.  It is neither.  We all manage our households every day of our lives.  Our time, our children.  With outputs much more unmeasurable than measurable.  Business, especially large ones, have a lot of distance between what their employees do and the final products produced.  As many management gurus will tell you, management is more art than science.  Supervisors have to evaluate the performance of subordinates every day, when the impact of the subordinate’s work on final products is tenuous at best.  All of these examples involve human judgment.  Sometimes measuring some outcomes can improve some assessments, but many times it can misdirect attention to what may be measurable but not most important.  If businesses had to manage solely or even primarily by focusing on measurable outputs, most would have gone out of business long ago.  There is a whole literature in the field of public administration pointing out the many circumstances in which focusing management on measurable output leads to distorted and inefficient decisions.

This is equally true in education.  Neoliberal, market fundamentalists like Patrinos have tried to convince us for 30+ years that testing and measurement is a cheap solution to most of our educational problems.  For 30+ years, we have increased educational testing and measurement around the world and our educational problems remain, or have increased.  No Child Left Behind (NCLB) raised measurement to a fetish in the U.S., but, as with similar attempts elsewhere around the world, all the resources go to building a better thermometer with little attention to the causes of the illness or resources to do something about it.  Moreover, the testing and measurement fetish has distorted education towards simplistic measurements of language and math achievement, neglect of other subjects, rampant teacher dissatisfaction, and damage to our children.

Of course, in some instances, measurements can be useful.  But they are far from necessary, and when useful, what is measured should usually only be a small piece of what is assessed.  Before neoliberal dominance took hold, in the U.S. there was a strong movement towards portfolios of student work as the essential ingredients for assessment, much of which required qualitative judgment to assess, not relying on test scores or even grades necessarily.  Universities did and still do, in some places, accept portfolios in order to make admission decisions.  Colleges like Reed and Antioch in the U.S. do not give grades, yet they manage very well, as do universities that offer admission to their graduates based on qualitative assessments.  In vaunted Finland, classroom teachers control assessment; many use portfolio assessment and do not give grades or tests, yet they turn out some of the best test-takers in the world.  There is nothing cost-effective about testing.  It is another “cheap” reform brought to you by people who are unwilling to put in the resources needed to improve education.  To argue how cost-effective even high-stakes testing is also neglects the significant psychological and material damage it does to so many children. 

While testing is a relatively low-cost reform, compared to the investment in teachers, learning materials, principals, and schools that are really needed to improve education, the costs of testing add up to big business for business.  So there is a huge lobbying effort by firms like Pearson to make sure quantitative assessments are an integral part of a renewed EFA and the newly developing Sustainable Development Goals (SDGs).  Of course, measurement fetishism goes way beyond education.  Currently the SDGs include 17 goals and 169 targets.  Each target may have multiple indicators.  Each indicator must be decomposed into many sub-indicators that show the extent to which the indicator is realized for multiple marginalized groups.  This is less a system for sane management and more likely to be a welfare program for researchers.  So many resources will be spent on trying to develop, measure, and collect data on what may literally be thousands of indicators that there may be few resources and little attention left over to actually do something about the problems indicated.

Diane Ravitch, the noted U.S. education historian and policy analyst asks:  “How did our elected officials become convinced that measurement and data would fix the public schools?”  In a later book she blames the business model that has so dominated education reform in the U.S. and worldwide:  “It means an obsessive devotion to testing, accountability, and data.  Devotees of the business approach like to say, ‘You measure what you treasure.’  Believing this, they have fastened a pitiless regime of testing on the nation’s schools that now reaches as low as kindergarten…[and even] prekindergarten.”  Ravitch goes on to argue that we can’t measure what we treasure most – such as human relationships or love of art and music – and education has been distorted by a narrow focus on measurement and accountability.  While other countries have not gone as far as NCLB, this is the direction in which we are heading.

Don’t get me wrong.  Indicators can be useful.  But our fetishism with measuring everything has gotten in the way of emphasizing what is needed even more than indicators.  The legitimacy of the processes, from the local to the global, in which quantitative indicators and qualitative information and judgments are used is key.  We too often treat policy analysis and policy making as a technical process.  It should not be.  Decision-making should rest on very messy, very participative, democratic processes that engage people from the local to the global.  We need to have more deliberative democratic approaches in which thoughtful human beings are able to combine quantitative and qualitative judgements to make sensible decisions, often in situations of conflicting interests.  Of course, measurement has a place, but to argue ‘if you can’t measure it, you can’t manage it” is belied by experience, research, and common sense.

Last modified on Wednesday, 21 October 2015

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.

Steve Klees

Steven J. Klees (sklees@umd.edu) is the R. W. Benjamin Professor of International and Comparative Education at the University of Maryland. He did his Ph.D. at Stanford University and has taught at Cornell University, Stanford University, Florida State University, and the Federal University of Rio Grande do Norte in Brazil.

Prof. Klees' work examines the political economy of education and development with specific research interests in globalization, neoliberalism, and education; the role of aid agencies; education, human rights, and social justice; the education of disadvantaged populations; the role of class, gender, and race in reproducing and challenging educational and social inequality; and alternative approaches to education and development.

 

Other blog posts

The Value and Values of Quality Public Services

Written by
on Thursday, 23 May 2013

Years of demagogic attacks on public services in some countries, even if the services are efficient and of good quality, has created a barrier to understanding the value of quality public services (QPS) [1]. In this sense, the notion of QPS is an easier concept to explain if there is a common understanding that collective solutions to many of the challenges facing society will be fairer and more efficient, than if everybody fends for themselves individually in the market.  …

Read more...

Hidden Markets in the Digital Age: Global Patterns in the Privatization of Education?

Written by
on Wednesday, 15 June 2016

Both in the U.S. and abroad, there exists a political movement in support of education reforms based on the logic of the market that assumes that business strategies can and should transfer to education. The rise of the private provision of public education services reflects such marketplace values as profit-making, outsourcing, limited government regulation, competition and choice. It also incorporates elements of government contracting and vouchers [1]. Digital education is part of this trend. Despite claims that digital education represents…

Read more...
blog archive