Assessment, Metrics & Rankings
Having been in academics for quite some time, I have seen fads come, go and stick. A recent fad is the obsession with assessment. As with many such things, assessment arrived with various acronyms and buzz words. Those more cynical than I would say that all acronyms of administrative origin (AAO) amount to B.S. But I would not say such a thing. I do, however, have some concern with the obsession with assessment.
One obvious point of concern was succinctly put by a fellow philosopher: “you don’t fatten the pig by weighing it.” The criticism behind this homespun remark is that time spent on assessment is time that is taken away from the actual core function of education, namely education. At the K-12 level, the burden of assessment and evaluation has become quite onerous. At the higher education level, the burden is not as great—but considerable time is spent on such matters.
One reply to this concern is that assessment is valuable and necessary: if the effectiveness (or ineffectiveness) of education is not assessed, then there would be no way of knowing what is working and what is not. The obvious counter to this is that educators did quite well in assessing their efforts before the rise of modern assessment and it has yet to be shown that these efforts have actually improved education.
Another obvious concern is that in addition to the time spent by faculty on assessment, a bureaucracy of assessment has been created. Some schools have entire offices devoted to assessment complete with staff and administrators. While only the hard-hearted would begrudge someone employment in these tough times, the toughness of the times should dictate that funding is spent on core functions rather than assessing core functions.
The reply to this is to argue that the assessment is more valuable than the alternative. That is, that funding an assessment office is more important to serving the core mission of the university than more faculty or lower tuition would be. This is, of course, something that would need to be proven.
Another common concern is that assessment is part of the micromanagement of public education being imposed by state legislatures (often by the very same people who speak loudly about getting government off peoples’ backs and protecting businesses from government regulation). This, some critics contend, is all part of a campaign to intentionally discredit and damage public education so as to allow the expansion of for-profit education.
The reply to this is that the state legislature has the right to insist that schools provide evidence that the (ever-decreasing) public money is being well spent. If the legislatures did show true concern for the quality of education and were devoted to public education, this reply would have merit.
Predating the current assessment fad is a much older concern with rankings. Recently I heard a piece on NPR about how Florida’s Board of Governors (the folks who run public education) is pushing Florida public universities to become top ranked schools. There are quite a few rankings, ranging from US News & World Report’s rankings to those of Kiplinger’s. Each of these has a different metric. For example, Kiplinger’s rankings are based on financial assessment. While it is certainly desirable to be well ranked, it is rather ironic that Florida’s public universities are being pushed to rise in the ranks at the same time that the state legislature and governor have consistently cut funding and proven generally hostile to public education. One unfortunate aspect of the ranking obsession is that Florida has adopted a performance based funding system in which the top schools get extra funding while the lower ranked schools get funding cut. Since the schools are competing with each other, some of the schools will end up lower ranked no matter how well they do—so some schools will get cuts, no matter what. This seems to be an odd approach: insisting on improvement while systematically making it harder and harder to improve.
This is also a problem with assessment. To return to that in the closing of this essay, a standard feature of assessment is that the results of the previous assessment must be applied to improve each academic program. That is, there is an assumption of perpetual improvement. Unfortunately, due to budget cuts, there is typically no money available for faculty salary increases. As such, the result is that faculty are supposed to better each year, but get paid less (since inflation and the cost of living increase reduces the value of the salary). As such, the system is such that perpetual improvement of faculty and schools is demanded, but there are no incentives or rewards—other than not getting fired or being the school to get the most cuts. Interestingly, the folks imposing this system are the same folks who tend to claim that taxation and government impositions kill the success of business. That is, if businesses have less money and are regulated too much by the state, then it will be bad. Apparently this view does not extend to education. But there might be an ironic hope: education is being “businessified” and perhaps once the transformation is complete, the universities will get the love showered on corporations.