Monday, March 21, 2011

Key Performance Indicators and the Heightening Contradictions of Academia

Chris Beneke

Of what value is your scholarship? Historians in Britain are receiving unsettling precise answers to that question. In case you hadn’t heard the news, British academics are now locked into a quality control regime that forces them to measure up against “Key Performance Indicators” over a 6 to 7 year span. The measures are largely determined by government officials though the actual measuring is done by historians.

Better minds have already suggested that the quantification of humanities scholarship through such mechanisms will dampen creativity, discourage ambitious long-term projects, and lower scholarly quality, while sucking much of the joy out of professional historical work. Randall blogged about funding-driven assessment tools a couple of weeks ago, Anthony Grafton’s AHA President’s column mentioned it in January, and Simon Head recently wrote a more extended analysis for the New York Review of Books. (I did some hand wringing myself in November.) Nonetheless, one paragraph in Simon Head’s account of the system struck me as especially noteworthy:

In the humanities the RAE (Research Assessment Exercise) bias also works in favor of the 180–200-page monograph, hyperspecialized, cautious and incremental in its findings, with few prospects for sale as a bound book but again with a good chance of being completed and peer-reviewed in time for the RAE deadline. A bookseller at Blackwell’s, the leading Oxford bookstore, told me that he dreaded the influx of such books as the RAE deadline approached.

In other words, the new British accountability systems seems perfectly designed to heighten the contradictions in the academic world, ensuring that the “research output” that scholars are dourly incentivized to produce is less accessible to the larger public and therefore less likely to contribute to the informed consideration of things that the public worries about—like, for instance, big social, political, and ethical problems. The argument can surely be made that rigorous research measures drive scholars toward more focused and more readily publishable research, which will ultimately makes a greater indirect contribution on the world. But I doubt that it would be convincing, especially when it comes to historical research.

Indeed, the RAE seems well designed to thwart scholars who believe that they should write engaging volumes that people who don’t care a whit about the distinction between social history and cultural history (the common term for them is “general readers”) might actually desire to read. Despite the precarious state of academic publishing, historians have enjoyed the indulgence of academic presses in recent years because of their faith that we will eventually write such books—if not the first time around, then the second, or third. It’s hard to imagine that system holding up if we’re flooding bookstore shelves with carefully calibrated units of research output, rather than, you know, books.


Randall said...

Chris: This state of affairs reminds me a little of the "best" intentions of No Child Left Behind and teaching students to pass tests. Reading Times Higher Ed makes me think there is a political battle going on here as well.

hcr said...

I'm still an agnostic. There is a problem being addressed by this and other efforts to get academics to produce something that can be measured. We do have a number of colleagues who don't teach much because they say they're doing research, but then don't produce anything either. This creates an unfair burden on those who really are doing both (or even one or the other). It doesn't seem to me unreasonable to ask academics to produce a significant piece of work every seven years or so. Now, what that should be is debatable. Timothy Gilfoyle has tried his hand at a list that might make more sense for historians than what seems to be the case in Chris's post. But I'm not ready to say that trying to get non-producers actually to do something will stifle creativity. It might increase it by opening up the field for more productive junior people.

Randall said...

Added to all the hubbub about assessment is the declining state of history in England. James Vernon writes in the latest issue of Perspectives on History:

"Even before the latest round of cuts imposed in England, the University of Sussex had restructured its impressively ranked history department by axing entire fields of study (along with the faculty who taught them)—British history before the 18th century and European history before the 20th century—because their low enrollments apparently made them unsustainable. Now the government has announced it will entirely cease to fund the teaching of arts, humanities, and social sciences at English universities from 2012. Historians in England must now defend the public utility of their discipline against a seemingly pervasive belief in market mechanisms."

Chris Beneke said...

I'd be interested to see Gilfoyle's list. I think accountability is essential and also believe that historians should be working very hard at teaching, service, and research. It's just that these productivity measures so often seem to emanate from non-humanities disciplines (or government mandates) where the scholarly standards aren't well adapted to distinguish between good, bad, and mediocre historical work. Ultimately, we're going to have to figure out how to make ourselves accountable, or let others do it for us.

Chris Beneke said...

Thanks for posting that Randall. The stuff happening in Britain should spur our thinking on this.

Randall said...

I think your right about the non-humanities assessment. Some of this seems to be coming from the corporate world.

dan allosso said...

But if programs are being cut back because of low enrollment, doesn't that mean that not only corporate types but also undergrads (the next generation of regular people coming up) don't see how the material being presented by historians is relevant and useful in their lives? It doesn't sound to me like the 180-200 page laser-focused monograph discussed above directly addresses this issue. But there seem to be unanswered questions about what (and how much) historians should be doing, especially if they are relying on "public" funding. That seems legit. Of course, when a monstrously mismanaged Wall Street gets a trillion dollar bailout so they can continue doing what they're doing, it's also fair to ask whether the scrutiny on humanities departments is reasonable, or maybe in some cases, politically motivated?

Chris Beneke said...

That's a good point, Dan. We also need to ask ourselves some probing questions about the value that we're providing in the classroom--or not providing, because we're not in the classroom, or because we are not sufficiently attentive to student learning there.

LD said...

I guess today is my day to think about academe as marketplace. It has been much on my mind -- not just from reading Age of Fracture and thinking, but also from hearing of recent efforts by my own university to improve our "customer service" to our student population.

But in thinking about academe as marketplace, Chris Beneke's comment here caught my attention: "We also need to ask ourselves some probing questions about the value that we're providing in the classroom."

We do need to think about this, always. And it's complicated.

I think it's unconscionable to ignore the economic realities facing our students. I suspect most of us teach a fair number of first-generation students from working-class backgrounds. I am not going to try to "convert" any of them to a career path in the humanities. There ain't one.

What I will do is try my best to make sure that I teach in such a way that they are able to derive both quantitative and qualitative value from the humanities. The humanities have practical purposes, and they also have the potential to give us a sense of life that is much more than a concern for practicality -- even if we are from a working-class background.

But I also think we need to push back -- hard -- against the corporate model of the university which fixes a literal "cash value" on ideas based on a market model which by definition cannot account for anything that money can't buy.

Here's my blog post about this from today. I have a feeling this academe-as-market idea is going to be bugging me for a while...
The Marketplace of Intellectuals