A Philosopher's Blog

Blackboard Lessons from the Semester

Posted in Universities & Colleges by Michael LaBossiere on June 19, 2013
Image representing Blackboard as depicted in C...

Image via CrunchBase

At the end of every semester, I engage in two types of assessment of my courses. The first is the bureaucratic assessment that I am required to do-this generates the right sort of numbers and properly buzzed words to be added to the volumes of data that seem to be studiously ignored by those who demand such tribute. The second is a thorough examination of what worked, what did not and what distinguished the two. The second assessment is used to improve each class, but does not contain the sort of data and buzzed-words that are considered the proper sort of assessment-that is, the sort that is beloved by the bureaucracy.

For the past few semesters, I have been evaluating and testing Blackboard-specifically the use of Blackboard as a means of evaluation and testing. On the plus side, Blackboard enables me to offload the mechanical aspect of teaching, namely the grading of true/false and multiple choice questions. On the minus side, the use of Blackboard does provide the students with new opportunities to cheat. Perhaps more importantly, there is the question of whether or not such evaluation is actually accurate in terms of properly assessing student ability.

In regards to cheating, I was surprised to find that the impact of cheating on Blackboard exams seems to be the same as the impact it has on in-class exams. This is based on comparing the scores on the Blackboard exams with those of the in class exams. The comparison shows that the results are quite similar, thus suggesting that the cheating methods used on Blackboard exams are used with about the same frequency and effectiveness as those used in traditional exams. Naturally, I am open to alternative explanations for the similarity in scores.

Because of budgetary concerns, the students do not have monitoring devices watching them take the exams. As such, I had to build the defense against cheating into the questions themselves. I was able to do this using information gathered from the take-home assignments I have been giving for years (which allowed the same opportunities to cheat that Blackboard based exams allow). The general idea is obvious enough: questions that require an understanding of the material make many methods of cheating (mainly looking through the notes or textbook) ineffective. However, students can still use other methods-such as getting a better student to take the exam or working together on the exams. While this no doubt occurs, it seems to be occurring on a relatively small scale or these methods actually create the same results as students taking the exams the “old fashioned” way. That is, the cheating is no more effective than the students taking the exams honestly.  I have this view because, as noted above, the exam scores for the Blackboard exams are essentially the same as those for traditional exams (that is, I get the expected bell curve).  Naturally, I could be in error here.

In addition to the education related goals, I also have the goal of reducing the amount of student problems I have to deal with, such as missed exams and the demand for make-ups. When I first tried Blackboard as a means of giving exams in my hybrid course, I was slightly surprised when the percentage of students “missing” a Blackboard exam turned out to be the same as the percentage who miss an in-class exam. In general, about 10-15% of the class missed each exam, regardless of whether it was on Blackboard or in the classroom.

I was slightly surprised by this result for two reasons. First, the Blackboard tests were available from day one of the semester and could be taken at anytime up until the deadline.  In contrast, in-class exams can only be taken in class on the set date. Second, the excuses I usually received from students regarding missing exams usually involved problems with attending the exam in person. However, the Blackboard exams can be taken from anywhere with access to the internet-and every student seems to have a smart phone, tablet, or laptop  (which they often use in class). As such, it seemed reasonable to expect that the percentage would be lower for the Blackboard exams.

But, a little reflection explained the lack of difference. First, students generally tend to wait to the last minute to do anything and hence they usually waited as late as possible to do the exams-sometimes too late, thus missing the deadline. Second, some of the factors that would impact being able to take an in-class exam would also impact being able to take an exam on Blackboard (such as attending a funeral or being sick). Interestingly, though, I found that although the percentage of students missing exams remained the same, the excuses changed: instead of funerals and illness preventing them from attending the exam ,dire (but often vague) technological problems prevented them from taking the exam before the deadline. As might be imagined, I could never confirm these technological problems. After all, I’m not going to get Google or the NSA to confirm for me that Sally Student’s apartment wi-fi went out one hour before the deadline for an exam or that Sam Student’s laptop “locked up” when he went to take the exam one hour before the deadline.

Since handling “make-up” exams on Blackboard is a bit annoying, I decided to try another method of reducing this problem. During the past semester (a six week summer semester) I set a due date for each exam at the end of the section the exam covered. The deadline for the exam was, however, set for the due date of the next exam. All the exams were accessible to the students from day one of the semester.

As I expected, this did not eliminate the problem. However, it reduced the percentage of missed exams from 10-15% down to about 5-7% for the semester. In these cases, there were two main categories for excuses given for missing an exam deadline: technological problems (combined with waiting to the deadline) and professions of ignorance regarding the deadline for an exam.  For example, I am writing this sentence at 5:21 pm on the day of the final for my summer class and I just received a panicked email from a student claiming that she did not know when the exam deadline was (it was at 3:00 pm). I put the date and time of the final on the board and announced it in class everyday  last week, plus Blackboard lists the deadline date and time as well. In terms of the professed ignorance excuse, I believe that I have done all that I can reasonably do to keep students informed. It would be interesting to learn how people form erroneous beliefs about deadlines-especially when the correct information is presented regularly and is readily available.

These findings did confirm some obvious things: 1) no matter what, some people will put things off to the last minute, 2) no matter how much effort a professor takes to inform students, some of them will remain in ignorance of deadlines, and 3) some people will simply let deadlines go by-most likely because they believe they will be able to do it after the deadline anyway. As with any such problem, the rational solution involves balancing the cost of reducing the problem with the reduction gained. At this point, I think I have done as well as I can on my end-that is, that any additional reduction of the problem would be a bigger problem than the problem.

While Blackboard did reduce some problems, it also created other difficulties. The first was not a problem with Blackboard, but merely a Blackboard variation on a problem with people. To be specific, some students claimed that Blackboard had cut off their exam early or “locked up” on them, when this had not, in fact, happened. Fortunately, Blackboard provides information that makes it possible to sort out such matters. To be fair, some students might have honestly mistaken the fact that Blackboard enforces the time deadlines as cases in which Blackboard was doing something wrong.

The second was a problem with Blackboard-or rather a problem for anything web-based. As might be guessed, students did encounter legitimate problems with browser crashes and other such things. As might also be guessed, I had informed the students to 1) not wait to the last minute to take an exam and 2) contact me as soon as the problem occurs so I can reset the exam before the deadline runs out. As might be guessed, the fact that people will ignore #1 can make it a problem for me to be able to reset an exam before time runs out.

I attempted to address this problem by having the due dates and deadlines mentioned above. I also set the exam deadlines at a time I would be awake and able to check my email regularly-this did dismay the students since they wanted the deadlines to all be a midnight. Not surprisingly, students did wait until the last hour to start an exam. For example, a student emailed me at 1:41 (by the gmail time stamp) to inform me that she got kicked out of the two hour final, which had a 3:00 pm deadline. The email did not reach the FAMU server until 2:12 PM and I saw it at 2:40. Since my only option is to clear an attempt, I offered to do so and to extend the final deadline for everyone in class (what I do for one, I must do for all) to allow her to complete the exam.

The obvious issue with a Blackboard deadline is that some people will push close to the deadline and then run into legitimate problems (like Blackboard or their browser giving them the boot). On the one hand, it is certainly reasonable to enforce the deadline-that is, reset the test for the student and given them until the deadline to complete it. After all, neither the deadline nor the possibility of problems should come as any surprise to the student.  Also, as some of my colleagues say, being held to a deadline is a good life lesson. On the other hand, it can be argued that the student is entitled to take the exam and that a problem that is not her fault should not deny her the chance to have the full time for the exam. Thus, the deadline should be extended.  Of course, there is then the matter of having a problem happen after the extension, which could keep extending the deadline. There is also the obvious point that waiting to the last minute is the student’s fault, even if the problem that he runs into is not his fault.  To use an analogy, if I wait to the last minute to drive to a race and miss it because I did not foresee the construction that required me to make  detour, the fault of missing the race would certainly seem to be my own.

My Amazon Author Page

Enhanced by Zemanta

6 Responses

Subscribe to comments with RSS.

  1. T. J. Babson said, on June 19, 2013 at 11:56 am

    Slightly off topic but still on the theme of teaching and education.

    Mike, I would be very interested to read your thoughts about student evaluations. Are these really a good guide to how well a professor is performing? Do Department heads and Deans take these too seriously? Should the students wait a year before evaluating the course?

    • Michael LaBossiere said, on June 19, 2013 at 2:47 pm

      Good questions.

      When I was a TA, the student evaluations at OSU included a blank sheet that allowed the students to make written comments. In some cases, the students just wrote silly things or drew cartoons, but in other cases I actually got some very useful feedback about my performance. For example, like most philosophers I tended to use rather unusual examples to illustrate points and one student made the excellent point that such wacko examples did not really help. S/he suggested using examples that were real and, if possible, within the realm of the students’ experience. Such substantive and considered feedback can be a good guide.

      However, when I started teaching at FAMU, the student evaluations did not include the blank sheet-just some generic ranking questions. These questions were later simplified and reduced in number to 8 (as I recall). As such, the data they yield seems rather limited. My evaluations are consistently good (B+ to A- range) but they do not give me a good guide to my performance: maybe the students just like me.

      What I mainly focus on is improvements in student performance in the class, especially when it comes to the papers. I also collect data via anonymous Google Doc surveys with questions that focus on my classes. Most students claim the classes helped them, but most also claim that they were already competent in the area of assessment.

      As might be guessed, there is a massive focus on assessment at colleges. Interestingly, the use of these evaluations seems to be rather limited-my university does not seem to use them.

      To directly focus on your questions, I would say that the student evaluations often reflect how the students feel about the class (and what grade they get) rather than how well the professor performed. After all, how often will students think “I got a C- in that class, but that was my fault-the professor did a great job”? There is also the question of the index of performance for professors. Graduation rate? Employment rate? Performance on standardized tests?

      Chairs and deans seem to vary a great deal in terms of the weight put on them. Some do not put any weight on them at all while others take them very seriously.

      I think if students waited a year, they would have totally forgotten the course-for the most part. Then again, I still remember the really good and really bad profs I had.

      I do think there is value in student feedback, but I believe that the actual implementation tends to be weak in regards to usefulness.

      • T. J. Babson said, on June 19, 2013 at 3:42 pm

        Do you think it would be better or worse to have peer review of teaching? How does your department evaluate the teaching of a professor going up for tenure?

        I think at Stanford they have a special office dedicated to peer review of teaching that is supposed be be normalized across the entire university.

        • Michael LaBossiere said, on June 19, 2013 at 3:55 pm

          It depends on the peers. When I was a grad student, the university offered (and required) a fair amount of evaluation by the faculty as well as by some office of teaching stuff (I can’t recall the exact name). This feedback was actually useful, especially seeing the films of me teaching and getting comments from experienced professors.

          When I went to FAMU, I did get evaluated by the chair, but peer review was very limited-mainly because of the crushing workload and the small size of the unit. This review was for tenure and promotion, but there was not much in the way of substantial feedback.

          I would favor peer review and feedback, but here is the practical dilemma: If the peer evaluation involved very little time and effort, faculty would do it, but it would have little value. If the peer evaluation was substantial, then the faculty would be doing yet another job for free.

  2. Nancy Matchett (@njmatch3) said, on June 23, 2013 at 11:35 am

    My findings, and conclusions, are similar to yours. But I have another interesting result. I have been teaching 2 sections of the same course — one online and one F2F – almost every term for 5 years now, and while the average student grade is quite similar across the two sections, I get a fairly standard bell curve distribution in the F2F section but an inverse bell curve in the online one. This persists despite various attempts on my part to move the distribution. My hypothesis is that students who select online courses tend to be either highly motivated, self-directed learners, or under-motivated/unprepared learners. Do you see anything in your use of Blackboard that might confirm this?

    • Michael LaBossiere said, on June 23, 2013 at 11:50 am

      That is an interesting finding.

      I have yet to teach a fully online class, which probably helps explain why my results are the same as with my previous classes-the change has been mainly in the mechanism for test taking.

      I have spoken to colleagues who have taught the fully online classes and their experiences match your own-they have said that. Seems like they get a more pronounced bimodal distribution. Some students are very active and responsible, while there is a second group who seem to regard an online class as a way to take it easy. They are, sadly, in error-online classes actually seem to be worse on slackers than traditional classes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: