A Philosopher's Blog

Assessment, Metrics & Rankings

Posted in Universities & Colleges by Michael LaBossiere on September 26, 2014

 

Having been in academics for quite some time, I have seen fads come, go and stick. A recent fad is the obsession with assessment. As with many such things, assessment arrived with various acronyms and buzz words. Those more cynical than I would say that all acronyms of administrative origin (AAO) amount to B.S. But I would not say such a thing. I do, however, have some concern with the obsession with assessment.

One obvious point of concern was succinctly put by a fellow philosopher: “you don’t fatten the pig by weighing it.” The criticism behind this homespun remark is that time spent on assessment is time that is taken away from the actual core function of education, namely education. At the K-12 level, the burden of assessment and evaluation has become quite onerous. At the higher education level, the burden is not as great—but considerable time is spent on such matters.

One reply to this concern is that assessment is valuable and necessary: if the effectiveness (or ineffectiveness) of education is not assessed, then there would be no way of knowing what is working and what is not. The obvious counter to this is that educators did quite well in assessing their efforts before the rise of modern assessment and it has yet to be shown that these efforts have actually improved education.

Another obvious concern is that in addition to the time spent by faculty on assessment, a bureaucracy of assessment has been created. Some schools have entire offices devoted to assessment complete with staff and administrators. While only the hard-hearted would begrudge someone employment in these tough times, the toughness of the times should dictate that funding is spent on core functions rather than assessing core functions.

The reply to this is to argue that the assessment is more valuable than the alternative. That is, that funding an assessment office is more important to serving the core mission of the university than more faculty or lower tuition would be. This is, of course, something that would need to be proven.

Another common concern is that assessment is part of the micromanagement of public education being imposed by state legislatures (often by the very same people who speak loudly about getting government off peoples’ backs and protecting businesses from government regulation). This, some critics contend, is all part of a campaign to intentionally discredit and damage public education so as to allow the expansion of for-profit education.

The reply to this is that the state legislature has the right to insist that schools provide evidence that the (ever-decreasing) public money is being well spent. If the legislatures did show true concern for the quality of education and were devoted to public education, this reply would have merit.

Predating the current assessment fad is a much older concern with rankings. Recently I heard a piece on NPR about how Florida’s Board of Governors (the folks who run public education) is pushing Florida public universities to become top ranked schools. There are quite a few rankings, ranging from US News & World Report’s rankings to those of Kiplinger’s. Each of these has a different metric. For example, Kiplinger’s rankings are based on financial assessment. While it is certainly desirable to be well ranked, it is rather ironic that Florida’s public universities are being pushed to rise in the ranks at the same time that the state legislature and governor have consistently cut funding and proven generally hostile to public education. One unfortunate aspect of the ranking obsession is that Florida has adopted a performance based funding system in which the top schools get extra funding while the lower ranked schools get funding cut. Since the schools are competing with each other, some of the schools will end up lower ranked no matter how well they do—so some schools will get cuts, no matter what. This seems to be an odd approach: insisting on improvement while systematically making it harder and harder to improve.

This is also a problem with assessment. To return to that in the closing of this essay, a standard feature of assessment is that the results of the previous assessment must be applied to improve each academic program. That is, there is an assumption of perpetual improvement. Unfortunately, due to budget cuts, there is typically no money available for faculty salary increases. As such, the result is that faculty are supposed to better each year, but get paid less (since inflation and the cost of living increase reduces the value of the salary). As such, the system is such that perpetual improvement of faculty and schools is demanded, but there are no incentives or rewards—other than not getting fired or being the school to get the most cuts. Interestingly, the folks imposing this system are the same folks who tend to claim that taxation and government impositions kill the success of business. That is, if businesses have less money and are regulated too much by the state, then it will be bad. Apparently this view does not extend to education. But there might be an ironic hope: education is being “businessified” and perhaps once the transformation is complete, the universities will get the love showered on corporations.

My Amazon Author Page

My Paizo Page

My DriveThru RPG Page

 

4 Responses

Subscribe to comments with RSS.

  1. T. J. Babson said, on September 26, 2014 at 7:01 pm

    “Interestingly, the folks imposing this system are the same folks who tend to claim that taxation and government impositions kill the success of business.”

    Mike, why do you keep forgetting that this system was pushed by Obama:

    Today, President Obama outlined an ambitious new agenda to combat rising college costs and make college affordable for American families. His plan will measure college performance through a new ratings system so students and families have the information to select schools that provide the best value. And after this ratings system is well established, Congress can tie federal student aid to college performance so that students maximize their federal aid at institutions providing the best value.

    http://www.whitehouse.gov/the-press-office/2013/08/22/fact-sheet-president-s-plan-make-college-more-affordable-better-bargain-

    • Michael LaBossiere said, on September 29, 2014 at 11:29 am

      My main focus was on a specific program in Florida which has nothing to do with Obama, but the discussion can be broadened to Obama’s proposal.

      My main concern about the Florida system is that no matter how well the state schools actually do, the schools that do less well are automatically punished. So, a school could be doing great, but as long as it is ranked lower than the other state schools it gets its funding cut. It also works the other way-all the schools could be doing badly, but the schools that do less bad would get rewarded.

      A fair rating system would be fine. My main concerns here are 1) whether the assessment system would be accurate and fair, 2) whether or not this would work as intended and 3) whether or not the system would penalize or reward based solely on place rather than quality.

      • T. J. Babson said, on September 29, 2014 at 11:37 am

        Mike, speaking of assessments, you might be interested in this piece from NPR on student assessments of teachers:

        http://www.npr.org/blogs/ed/2014/09/26/345515451/student-course-evaluations-get-an-f

        My favorite bit:

        Michele Pellizzari, an economics professor at the University of Geneva in Switzerland, has a more serious claim: that course evaluations may in fact measure, and thus motivate, the opposite of good teaching.

        His experiment took place with students at the Bocconi University Department of Economics in Milan, Italy. There, students are given a cognitive test on entry, which establishes their basic aptitude, and they are randomly assigned to professors.

        The paper compared the student evaluations of a particular professor to another measure of teacher quality: how those students performed in a subsequent course. In other words, if I have Dr. Muccio in Microeconomics I, what’s my grade next year in Macroeconomics II?

        Here’s what he found. The better the professors were, as measured by their students’ grades in later classes, the lower their ratings from students.

        “If you make your students do well in their academic career, you get worse evaluations from your students,” Pellizzari said. Students, by and large, don’t enjoy learning from a taskmaster, even if it does them some good.

        • Michael LaBossiere said, on September 30, 2014 at 4:05 pm

          Students do tend to rank hard teachers lower. That said, some students do value teachers who actually teach them.

          Having looked at some of the comments on Rate My Professor, student evaluations seem to be rather a mixed bag.

          Back in the day, the students got a blank sheet of paper to write out comments and we got the papers back the next quarter. Some of the comments were things like “I didnt like the teacher because he made me write lots of papers” or “the class suck because I had to take it.” Others were actually useful critiques. My two favorites though, were an illustrated poem about a bird and a hand drawn cartoon about a classroom full of squirrels.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: