Checklists and transparency

In an excellent comment on my first checklist post, Bryce Bunting wrote:

On another note, I could see educators voicing arguments similar to those of physicians with regard to why they don't need checklists (e.g. "teaching is an art," "I don't want to be restricted by a checklist," etc.).

Shortly after reading Bryce's comment I got a copy of an article on health care reform from my boss. (Thanks, Cid.) The piece, Making Health Care Better profiles Intermountain Health Care's efforts to improve health care and cut costs (go, Utah!)

IHC uses checklists (or protocols like them) but the improvement of health care system-wide comes not just from the checklists but from using and sharing data from them. IHC's computer system recommends certain treatments to doctors and nurses (the preferred treatments are developed by a team of doctors, nurses, and administrators from IHC). They are free to use those treatments or choose others. Those choices and the outcomes go back into the system. So, over time, IHC has developed a set of preferred practices, based on research and outcomes, and the evidence from those practices is visible.

Contrast this with what happens in education. Teachers prepare for class, using a combination of the things they have done in the past, innovations they prefer, ideas from colleagues, etc. My sense is that most teachers (and nearly all professors) spend a substantial amount of time in preparation, as well we should.

Then they hold class. There, they likely do most of the things they planned to do, leaving some out because of time constraints or because the class goes in a different direction. At some point students are supposed to show evidence that they learned. The professors evaluate them. Class ends. Students get final grades and go on to the next course.

The major difference between excellent health care and excellent college classes, then, isn't the preparation. It probably isn't the presence or absences of checklists either (though they cannot hurt, it seems). Instead it is transparency.

The transparency problem has two components. First, with the exception of student teachers and faculty undergoing review, no other qualified teacher ever sees what happens in a classroom. Second, no one ever sees the outcomes of what happens in a particular class. As a result, systemic improvement generally depends on training rather than results.

Why is the classroom so opaque? Tradition has something to do with it--universities still carry with them a model of master-disciple born in the middle ages, and that model suggests that the key relationship is between teacher and student, not between student and educational system. Academic freedom, at least as we have come to see it, influences as well.

But I think the main obstacles are not cultural, but systemic. After all, faculty favor collaboration with colleagues on nearly every other task (committees, research, writing, etc.) and most good teachers turn to colleagues for guidance all the time. The "fear" of having someone else watch and comment on your work is probably not too deep, particularly if that someone is a fellow faculty member.

But the system makes it impossible in three ways: first, almost no system of education believes in standard practices (a "meta-checklist" if you will). Teaching chemistry differs from teaching history; teaching first-year students isn't like teaching grad students. But why is this? Those few institutions with a campus-wide commitment to particular teaching approaches seem to have positive effects. Second, we schedule faculty work so nearly all professors are teaching simultaneously. Why don't colleagues visit your classes? Because they are in their own. Third, we don't know what outcomes would look like. What would be the result of a well-taught class in history? How would it differ from a similar course in, say, music?

I don't have good solutions to these transparency problems. But I can think of a simple first step. As part of campus faculty development efforts and accounted for in faculty workload, mandate (or incentivize) faculty members to enroll in one class taught by a colleague each year. A few benefits would be almost immediate. First, faculty development would be campus-wide rather than focused on the willing few. Second, faculty would learn from each other. Third, they would see the experience of students first-hand. And fourth, colleagues would be in a position to talk with each other about the best ways to do our common work.