I have tried to keep editorializing about Westminster College in the background of this blog. So while I've referred to lots of things going on here, I haven't tried to make the case that Westminster is unusual or wonderful (though in many ways it is).
But this article, reporting views on assessment and learning in higher ed held by national policy wonks, makes me want to brag a bit. They argue that while good assessment practice is pretty well understood, it tends to be isolated, a department here or there doing good work, and its findings hidden from the public. And they pine for the day when clear outcomes (preferably national) and transparency bring about a major shift in higher ed and graduation rates. Absent these things, we can never have real accountability in education, they argue.
Their doubts about assessment's progress may describe many places, but Westminster isn't one of them. The college established learning goals for all students 6 years ago. All academic and co-curricular programs now connect their activities to the learning goals, report each year on their progress, and revise their activities in response to findings.
The college tracks all sorts of institution-wide data, using NSSE, the CLA, and a range of in-house surveys. These data, and those gathered in programs, are analyzed by campus leaders twice a year, each time in an effort to find areas for improvement. Our NSSE data are posted publicly on our website. We also track all first-year students, and closely watch retention rates over time.
We are now setting up an e-portfolio program which, if it is entirely implemented, would ask each undergraduate student to complete an e-portfolio for graduation. The portfolio would allow students to see their own growth and achievement of learning goals over time. And it would allow us to certify that all our students achieve the goals.
So in all sorts of ways Westminster follows best practices. We use national and internal standards to measure ourselves. We tie curriculum and co-curriculum to learning goals. We revise our programs in response to data. And we are working to ensure that all students who graduate from Westminster get a meaningful education based in best practices and our own culture.
These efforts have brought about enormous good on campus. The language of learning is now common. Key faculty lead the assessment effort. Students are familiar with the learning goals and are increasingly committed to achieving them. And having a set of campus-wide outcomes allows us to discuss ways to rethink education to have an ever-greater focus on learning and an ever-smaller cost.
At the same time, our outcomes, learning, and assessment efforts are no panacea. Many courses are still teacher-, not learner-driven. We are often buried in data. Just as often we make changes in response to relatively small shifts in NSSE or other data. And perhaps most importantly, with all of our efforts, we still don't retain more than 79% of our students from the first to the second year. And of the students we retain, many graduate with a credential, not an education.
These two things--our retention rate and the characteristics of our graduates--suggest to me that the next frontier for assessment has little to do with what the college does. (After all, this sort of college and program-focused assessment grows out of an input-based model, one that undergirds the instruction paradigm.) Instead, it must be focused on what students bring to their learning. What are their motivations? Their strengths? Their desires? The obstacles to achieving their goals and ours?
Absent these sorts of data, the best assessment is incomplete. And accountability cannot be fairly parcelled out--a portion to the college, a portion to its employees, and a portion to the most important participants in education--the students.
Thursday, January 28, 2010
Monday, January 25, 2010
The future of higher ed--institution types; educational characteristics
Our campus is ever more intently thinking about the future of higher education, and more particularly, what it means for Westminster College. Here is my shot at looking 20 years down the road and trying to figure out what the higher ed landscape will look like. (Please note that I am generally horrible at predicting the future, even of things that I have a big role in deciding, like what kind of car I will drive or what I want to order at a restaurant. So don't bet on what follows.)
Institution Types
Today, colleges and universities are defined by their size, the degrees they grant, and the role of research in setting institutional goals. Within 20 years, none of these will define the type of institution. Instead, they will be organized by the student's relationship with the institution. Here are four types I expect to see flourish:
Even though these seem like vastly different types of institutions, they share a number of characteristics which would allow one entity to be the sponsor for all of them. In other words, I'm thinking that one organization--a university--would be the umbrella under which some assortment of these institutions would be created, evaluated, improved, etc. Here are the common characteristics:
Institution Types
Today, colleges and universities are defined by their size, the degrees they grant, and the role of research in setting institutional goals. Within 20 years, none of these will define the type of institution. Instead, they will be organized by the student's relationship with the institution. Here are four types I expect to see flourish:
- Residential College--Parents of teenagers, and adults seeking education, will continue to choose a residential learning option. And educators will want to continue to provide it, because residential learning provides a setting for the development of the whole person, an opportunity for immersion, and the grounds on which a person can integrate the components of her life. This doesn't mean, though, that all (or most) students will live on campus for two semesters at a time. Instead, I expect that residential schools will look more like monasteries or retreat centers. Some people will come and immerse themselves for many years at a time, working at the school and becoming part of its community. Others will continue the existing model. And still more will come for very intense short-term learning experiences--a week or a month at a time.
- Training Schools--These institutions will be distinguished by their ability to provide professional training to students of all sorts. Some of it will be traditional training--auto mechanics, dentistry, MBAs. But there will also be types of training that we haven't always thought of as training. Employers tell AACU, for example, that they want employees who can think critically and work in teams and behave ethically. Training schools will provide programs in these areas as well--a mix of classes and experiences that add up to meaningful skills in well-defined topics.
- On-demand Learning--On-demand learning schools will look a lot like the educational components of YouTube, coupled with the customized products of Etsy.com. Students will decide what they want to learn--how to bake a cake, how to play guitar, how to understand what a top quark does--and teachers (most likely on a free-lance basis) will compete to teach them. A bunch of the education will be simply providing good content. But where teachers are needed they will act as tutors, interrogators, and scolds, pushing their students to do their best work.
- Further Learning--In the future of higher education, individual and (more likely) their employers will see the need to have focused, specialized opportunities to explore the edges of their fields. These opportunities will take place in R&D Centers sponsored by universities. There, cutting-edge experts and leaders from outside higher ed will work together on innovations that would be unlikely in a corporate environment. Think Aspen Institute that actually does something.
Even though these seem like vastly different types of institutions, they share a number of characteristics which would allow one entity to be the sponsor for all of them. In other words, I'm thinking that one organization--a university--would be the umbrella under which some assortment of these institutions would be created, evaluated, improved, etc. Here are the common characteristics:
- Personalization--regardless of the institution type, the educational program will be personalized for the student or group of students. Curricula will be revised to meet the needs and desires of individual students; courses (or whatever they look like--projects perhaps?) will roll out with students shaping the assignments; and "graduation" will come when individual students meet the goals they have jointly established with their faculty.
- Partnership--all of these institutions will be based on partnership relationships--between students and faculty, programs and employers, institutions and communities, experts and investigators, etc.
- Focus--No institution will be able to provide the full range of personalized learning opportunities to all students. So institutions will focus--with clear missions and clear areas of specialty. There will be no such thing as a "university" in other words, but students will have a real choice between institutions. And institutions will be able to distinguish themselves based on vision, curricula, and expertise, rather just than marketing, branding, price discounts, and impressive campuses.
- Creativity--All learning will be judged by outcomes, and all key outcomes will be demonstrated by students, in partnership with faculty, employers, and community, creating things. So if your course of student draws on a great deal of online content, you will complete your education by creating better online content.
- Technology--Technology will be everywhere, but it won't be seen as a solution. Instead, the use of technology will be a craft--something that requires skill, a sense of what is appropriate for the particular project and context. So in the same school, and maybe in the same program, one student will use on set of technologies, an another another. But the educational techno-future, where all content and interaction are on-line or technology-mediated, will be seen as too blunt a tool to create a meaningful education.
Thursday, January 21, 2010
Follow up on Haiti--do people only self-organize on the internet?
This article asks a much better informed set of questions about why aid has been so slow getting to Haiti. The conclusion: that the US is much more concerned about command and control than about distributing aid.
Why? In part because we have asked our military to deliver aid, but the military is set up to do command and control, so that is what they do. And in part because the presumption of elites (argues Rebecca Solnit in A Paradise Built in Hell) that in these situations, poor people and people of color tend to panic and therefore security is more important than aid.
It is obvious to me that people self-organizing for their own well-being predates the internet (Democracy in America, anyone?) and takes place without needing help from the state. Why assume that people in their extremity will not turn to each other for help? And why not, then, help them?
Why? In part because we have asked our military to deliver aid, but the military is set up to do command and control, so that is what they do. And in part because the presumption of elites (argues Rebecca Solnit in A Paradise Built in Hell) that in these situations, poor people and people of color tend to panic and therefore security is more important than aid.
It is obvious to me that people self-organizing for their own well-being predates the internet (Democracy in America, anyone?) and takes place without needing help from the state. Why assume that people in their extremity will not turn to each other for help? And why not, then, help them?
The power (?) of incentives in changing faculty behavior
I am at the AACU Conference in Washington DC. I spent yesterday at a pre-conference gathering of the Leadership Coalition, a consortium of 50 or so colleges working on "transformational change" initiatives, esp. those that focus on developing the whole student and ensuring a broad view of student well-being.
The Coalition has sponsored a survey of faculty to see what sort of innovations they prefer and why. They reported findings from a pilot survey yesterday. (They are doing a bigger survey using the same tool this spring.) Two interesting things:
1. 87% of faculty report changing their syllabi, and 34% report adopting a new pedagogy at least once a year. I'm not sure if this is a sign of innovation, but it does mean faculty play with the content of their courses more than their approach to teaching.
2. When asked what sort of incentives exist on their campuses to support innovation in teaching, faculty said the most common was "a culture that values excellent teaching" followed by "recognition (i.e. teaching awards)," "administrative support," and at the bottom, "course release" and "stipends". When asked what sort of incentive would get them to change their teaching, by far the two most powerful were "stipends (76% of faculty agreed)" and "course release (73% of faculty agreed)." The least influential, by far, were teaching awards (33%)
Now this second finding alone is unsurprising, even if it is a bit disappointing. (Here I will refer you yet again to two of my favorite TED talks: this one by Barry Schwartz and this one by Dan Pink both questioning the power of financial incentives to bring about positive, lasting change.) Its implication is that faculty will change their behavior, for a time, for between $500 and $3000 dollars.
But one further bit of data was more interesting: there is no correlation between the existence of stipends (or course releases) and faculty satisfaction or stress level. Nor is there a correlation between awards and satisfaction. Or, put another way, a stipend might change faculty behavior for a while, but it won't make faculty feel better about their work.
Why? Because stipends really function as a way to get faculty to temporarily change their priorities in a context where they already feel overwhelmed. But they do nothing to respond to that feeling of being swamped. (In fact, they might heighten that feeling.)
Interestingly, faculty who frequently revise their syllabi, or who adopt new pedagogies, do score higher on satisfaction. And, as the overall number of types of support for innovation increases, faculty feel more satisfied and flourish more. Why is this? Two theories (these are mine, though influenced by the conversation at the session): 1. that course revisions are signs of faculty control of their own core activities, and 2. the number of types of support for innovation is a sign that the institutional culture values the creative work of faculty.
What does this mean for people responsible for faculty development or change on campuses? Some simple but important things:
stay busy--providing lots of development options not only increases the possibility that you will influence more faculty but it also makes faculty as a whole feel like the institution values their work
start where the faculty are--find out what sorts of changes are already going on, and support them. Doing so builds on the intrinsic motivation that faculty already have. It also means that healthy faculty development will be obvious when small groups of faculty have chosen to work together (with institutional support) on topics of their own choosing. And a healthy campus culture will be when a large proportion of faculty are involved in these small groups.
Not rocket science I know. But also not general practice, which is instead characterized by a single approach to faculty development (ie workshops) and a set of initiatives that may or may not appeal broadly to faculty.
The Coalition has sponsored a survey of faculty to see what sort of innovations they prefer and why. They reported findings from a pilot survey yesterday. (They are doing a bigger survey using the same tool this spring.) Two interesting things:
1. 87% of faculty report changing their syllabi, and 34% report adopting a new pedagogy at least once a year. I'm not sure if this is a sign of innovation, but it does mean faculty play with the content of their courses more than their approach to teaching.
2. When asked what sort of incentives exist on their campuses to support innovation in teaching, faculty said the most common was "a culture that values excellent teaching" followed by "recognition (i.e. teaching awards)," "administrative support," and at the bottom, "course release" and "stipends". When asked what sort of incentive would get them to change their teaching, by far the two most powerful were "stipends (76% of faculty agreed)" and "course release (73% of faculty agreed)." The least influential, by far, were teaching awards (33%)
Now this second finding alone is unsurprising, even if it is a bit disappointing. (Here I will refer you yet again to two of my favorite TED talks: this one by Barry Schwartz and this one by Dan Pink both questioning the power of financial incentives to bring about positive, lasting change.) Its implication is that faculty will change their behavior, for a time, for between $500 and $3000 dollars.
But one further bit of data was more interesting: there is no correlation between the existence of stipends (or course releases) and faculty satisfaction or stress level. Nor is there a correlation between awards and satisfaction. Or, put another way, a stipend might change faculty behavior for a while, but it won't make faculty feel better about their work.
Why? Because stipends really function as a way to get faculty to temporarily change their priorities in a context where they already feel overwhelmed. But they do nothing to respond to that feeling of being swamped. (In fact, they might heighten that feeling.)
Interestingly, faculty who frequently revise their syllabi, or who adopt new pedagogies, do score higher on satisfaction. And, as the overall number of types of support for innovation increases, faculty feel more satisfied and flourish more. Why is this? Two theories (these are mine, though influenced by the conversation at the session): 1. that course revisions are signs of faculty control of their own core activities, and 2. the number of types of support for innovation is a sign that the institutional culture values the creative work of faculty.
What does this mean for people responsible for faculty development or change on campuses? Some simple but important things:
stay busy--providing lots of development options not only increases the possibility that you will influence more faculty but it also makes faculty as a whole feel like the institution values their work
start where the faculty are--find out what sorts of changes are already going on, and support them. Doing so builds on the intrinsic motivation that faculty already have. It also means that healthy faculty development will be obvious when small groups of faculty have chosen to work together (with institutional support) on topics of their own choosing. And a healthy campus culture will be when a large proportion of faculty are involved in these small groups.
Not rocket science I know. But also not general practice, which is instead characterized by a single approach to faculty development (ie workshops) and a set of initiatives that may or may not appeal broadly to faculty.
Monday, January 18, 2010
Haiti
Ok, I am out of my area of expertise, and driven by anger and frustration as much as anything, but explain to me again why aid has taken so long to get to Haitians?
My real question is this: in the immediate aftermath of the quake, when as one NPR reporter described it, the response amounted to "Haitians helping Haitians" why was the emphasis on creating the infrastructure necessary to get organized aid to Haiti, rather than getting the food and tools that Haitians could use to Haitians? Why was it not possible to drop food, shovels, and medicine across the city?
And why, today, five days later, when Haitians are running out of the strength and the goodwill to help each other, is the focus still on infrastructure? Do we not trust that humans, in the worst conditions, rise to be their best selves? Is it really so important to have state-certified order before help comes?
Think about more comprehensible disasters--a car crash for example. First responders are usually untrained people who pull over and try to help. That was happening in Haiti, in the worst conditions, for several days. And official aid givers seem to have been unable to take advantage of that good will. Why?
My real question is this: in the immediate aftermath of the quake, when as one NPR reporter described it, the response amounted to "Haitians helping Haitians" why was the emphasis on creating the infrastructure necessary to get organized aid to Haiti, rather than getting the food and tools that Haitians could use to Haitians? Why was it not possible to drop food, shovels, and medicine across the city?
And why, today, five days later, when Haitians are running out of the strength and the goodwill to help each other, is the focus still on infrastructure? Do we not trust that humans, in the worst conditions, rise to be their best selves? Is it really so important to have state-certified order before help comes?
Think about more comprehensible disasters--a car crash for example. First responders are usually untrained people who pull over and try to help. That was happening in Haiti, in the worst conditions, for several days. And official aid givers seem to have been unable to take advantage of that good will. Why?
schools as commemorators
With a couple of exceptions, all of the major commemorations of Martin Luther King Jr in Utah are affiliated with colleges and universities. This seems normal--after all, schools have become the main site of commemoration in American culture--President's Day, Constitution Day, and many smaller acts of public memory happen in schools or they happen not at all.
But it is strange as well, both for commemoration in general, and for MLK Jr Day in particular. Schools have been sites of commemoration forever, but until relatively recently, they were secondary sites. Fraternal organizations, churches, civic clubs, political parties and other groups in civil society handled local commemorations (or local versions of national commemorations), while the mainstream media presented a particular view of national issues.
This was the case even as recently as the 1960s, when King came to prominence. Major newspapers reported on the "I Have a Dream" speech and other major events in his life with remarkable uniformity. And after his death, the big commemorations, especially those calling for the creation of MLK Day, were driven by unions, churches, and civil rights organizations. Universities and schools were of minor significance.
It is easy to understand why universities were late to King commemoration. By the end of King's life, university students tended to see him as a sell-out. The Student Non-Violent Coordinating Committee had split with King's SCLC. University students who had been part of the civil rights movement in the mid-60s had either turned toward Black Power or towards anti-war efforts.
At the same time, public schools found King's legacy to be too controversial to attend to.
The change came about during the Reagan administration, which dragged its feet while the media and civil rights establishment turned their efforts towards the creation of Martin Luther King Day. Representatives from the Reagan administration did not want to appear to be too closely aligned with civil rights activists, so as King Day became a reality, they commemorated the day by visiting schools and reading to kids. A diminished view of King, one comfortable for public school consumption, emerged from the 80s and the effort to create a version of King that could win support for MLK Day.
As civil rights moved to the back burner as a political issue (and campus activism declined) universities began to embrace MLK Day as a way of commemorating their former fervor (hence the marches that characterize many MLK Day celebrations today). Once service-learning became a major force on college campuses, those marches were often accompanied by days of service, and the contemporary landscape of commemorations was set--recalling the I Have a Dream speech in public schools, service and marches in higher ed.
It is worth asking whether this is a good outcome. I am saddened that schools have become the guardians of King's legacy, if only because it implies that King has become part of our history, rather than a living part of our culture (and that our civil society has shrunken). I am a troubled that the main commemorations of his life are community service and marches. For while those were part of the repertoire of the civil rights movement, they were only a minor part. It was the speeches, the moral fervor, the religious vision, the legal acumen, and the relentless organizing that brought about the changes of the civil rights movement. Where are the institutions to commemorate those things?
But it is strange as well, both for commemoration in general, and for MLK Jr Day in particular. Schools have been sites of commemoration forever, but until relatively recently, they were secondary sites. Fraternal organizations, churches, civic clubs, political parties and other groups in civil society handled local commemorations (or local versions of national commemorations), while the mainstream media presented a particular view of national issues.
This was the case even as recently as the 1960s, when King came to prominence. Major newspapers reported on the "I Have a Dream" speech and other major events in his life with remarkable uniformity. And after his death, the big commemorations, especially those calling for the creation of MLK Day, were driven by unions, churches, and civil rights organizations. Universities and schools were of minor significance.
It is easy to understand why universities were late to King commemoration. By the end of King's life, university students tended to see him as a sell-out. The Student Non-Violent Coordinating Committee had split with King's SCLC. University students who had been part of the civil rights movement in the mid-60s had either turned toward Black Power or towards anti-war efforts.
At the same time, public schools found King's legacy to be too controversial to attend to.
The change came about during the Reagan administration, which dragged its feet while the media and civil rights establishment turned their efforts towards the creation of Martin Luther King Day. Representatives from the Reagan administration did not want to appear to be too closely aligned with civil rights activists, so as King Day became a reality, they commemorated the day by visiting schools and reading to kids. A diminished view of King, one comfortable for public school consumption, emerged from the 80s and the effort to create a version of King that could win support for MLK Day.
As civil rights moved to the back burner as a political issue (and campus activism declined) universities began to embrace MLK Day as a way of commemorating their former fervor (hence the marches that characterize many MLK Day celebrations today). Once service-learning became a major force on college campuses, those marches were often accompanied by days of service, and the contemporary landscape of commemorations was set--recalling the I Have a Dream speech in public schools, service and marches in higher ed.
It is worth asking whether this is a good outcome. I am saddened that schools have become the guardians of King's legacy, if only because it implies that King has become part of our history, rather than a living part of our culture (and that our civil society has shrunken). I am a troubled that the main commemorations of his life are community service and marches. For while those were part of the repertoire of the civil rights movement, they were only a minor part. It was the speeches, the moral fervor, the religious vision, the legal acumen, and the relentless organizing that brought about the changes of the civil rights movement. Where are the institutions to commemorate those things?
Friday, January 15, 2010
metaphors for disaster--eating the seed corn
Earlier this week University of Utah President Michael Young appeared before a legislative committee considering additional cuts to the higher education budget. Young told committee members that an additional 4% cut to state funding for the U (and the rest of the Utah system) would be akin to "eating our seed corn." "Twenty years from now," he asked, "do we want to have terrific prisons and terrible universities?"
Setting aside the fact that Utah's prisons are about as dismal as prisons in every other state (and that the Department of Corrections faces budget cuts as well), and that systems of higher ed in other states are facing fiscal problems larger than those in Utah, it is worth wondering what Young could have meant when he said that budget cuts would be like "eating the seed corn."
The most literal reading would be that by cutting budgets now, the university will starve in seasons to come. Read more liberally, a short-term solution, like cutting budgets, would push today's problem into the future. The question is whether by either of these readings, Young accurately describes the potential impact of this year's budget cuts.
The answer has to be no, for two reasons. From one perspective, Utah ate the seed corn long ago (or never had any at all). By many measures, the state of Utah already faces an education crisis. Most campuses are full to capacity, first- to second-year retention rates are between 43% (at UVU) to 83% (at the University of Utah) for four-year institutions (these data are from before the economic downturn), and a large portion of high school graduates never go to college at all. While a 4% cut may make things marginally worse, from an access to education perspective, things are already bad.
At the same time, a view from the outside suggests that a budget cut doesn't necessarily doom an institution for the long-term. City, county, and state governments have already sustained such cuts, but we don't hear anyone suggesting that Salt Lake City faces a long, bleak future because of those cuts. Businesses regularly face decreases in income and respond.
So this comes around to a more fundamental question--are institutions of higher education unlike other institutions, in that they cannot adapt to vagaries in funding? If so, why? And, more importantly, is greater funding necessary in order not just to maintain the status quo, but to actually reach those thousands of students who don't have access to higher ed in Utah?
Setting aside the fact that Utah's prisons are about as dismal as prisons in every other state (and that the Department of Corrections faces budget cuts as well), and that systems of higher ed in other states are facing fiscal problems larger than those in Utah, it is worth wondering what Young could have meant when he said that budget cuts would be like "eating the seed corn."
The most literal reading would be that by cutting budgets now, the university will starve in seasons to come. Read more liberally, a short-term solution, like cutting budgets, would push today's problem into the future. The question is whether by either of these readings, Young accurately describes the potential impact of this year's budget cuts.
The answer has to be no, for two reasons. From one perspective, Utah ate the seed corn long ago (or never had any at all). By many measures, the state of Utah already faces an education crisis. Most campuses are full to capacity, first- to second-year retention rates are between 43% (at UVU) to 83% (at the University of Utah) for four-year institutions (these data are from before the economic downturn), and a large portion of high school graduates never go to college at all. While a 4% cut may make things marginally worse, from an access to education perspective, things are already bad.
At the same time, a view from the outside suggests that a budget cut doesn't necessarily doom an institution for the long-term. City, county, and state governments have already sustained such cuts, but we don't hear anyone suggesting that Salt Lake City faces a long, bleak future because of those cuts. Businesses regularly face decreases in income and respond.
So this comes around to a more fundamental question--are institutions of higher education unlike other institutions, in that they cannot adapt to vagaries in funding? If so, why? And, more importantly, is greater funding necessary in order not just to maintain the status quo, but to actually reach those thousands of students who don't have access to higher ed in Utah?
Thursday, January 7, 2010
Checklists and transparency
In an excellent comment on my first checklist post, Bryce Bunting wrote:
On another note, I could see educators voicing arguments similar to those of physicians with regard to why they don't need checklists (e.g. "teaching is an art," "I don't want to be restricted by a checklist," etc.).
Shortly after reading Bryce's comment I got a copy of an article on health care reform from my boss. (Thanks, Cid.) The piece, Making Health Care Better profiles Intermountain Health Care's efforts to improve health care and cut costs (go, Utah!)
IHC uses checklists (or protocols like them) but the improvement of health care system-wide comes not just from the checklists but from using and sharing data from them. IHC's computer system recommends certain treatments to doctors and nurses (the preferred treatments are developed by a team of doctors, nurses, and administrators from IHC). They are free to use those treatments or choose others. Those choices and the outcomes go back into the system. So, over time, IHC has developed a set of preferred practices, based on research and outcomes, and the evidence from those practices is visible.
Contrast this with what happens in education. Teachers prepare for class, using a combination of the things they have done in the past, innovations they prefer, ideas from colleagues, etc. My sense is that most teachers (and nearly all professors) spend a substantial amount of time in preparation, as well we should.
Then they hold class. There, they likely do most of the things they planned to do, leaving some out because of time constraints or because the class goes in a different direction. At some point students are supposed to show evidence that they learned. The professors evaluate them. Class ends. Students get final grades and go on to the next course.
The major difference between excellent health care and excellent college classes, then, isn't the preparation. It probably isn't the presence or absences of checklists either (though they cannot hurt, it seems). Instead it is transparency.
The transparency problem has two components. First, with the exception of student teachers and faculty undergoing review, no other qualified teacher ever sees what happens in a classroom. Second, no one ever sees the outcomes of what happens in a particular class. As a result, systemic improvement generally depends on training rather than results.
Why is the classroom so opaque? Tradition has something to do with it--universities still carry with them a model of master-disciple born in the middle ages, and that model suggests that the key relationship is between teacher and student, not between student and educational system. Academic freedom, at least as we have come to see it, influences as well.
But I think the main obstacles are not cultural, but systemic. After all, faculty favor collaboration with colleagues on nearly every other task (committees, research, writing, etc.) and most good teachers turn to colleagues for guidance all the time. The "fear" of having someone else watch and comment on your work is probably not too deep, particularly if that someone is a fellow faculty member.
But the system makes it impossible in three ways: first, almost no system of education believes in standard practices (a "meta-checklist" if you will). Teaching chemistry differs from teaching history; teaching first-year students isn't like teaching grad students. But why is this? Those few institutions with a campus-wide commitment to particular teaching approaches seem to have positive effects. Second, we schedule faculty work so nearly all professors are teaching simultaneously. Why don't colleagues visit your classes? Because they are in their own. Third, we don't know what outcomes would look like. What would be the result of a well-taught class in history? How would it differ from a similar course in, say, music?
I don't have good solutions to these transparency problems. But I can think of a simple first step. As part of campus faculty development efforts and accounted for in faculty workload, mandate (or incentivize) faculty members to enroll in one class taught by a colleague each year. A few benefits would be almost immediate. First, faculty development would be campus-wide rather than focused on the willing few. Second, faculty would learn from each other. Third, they would see the experience of students first-hand. And fourth, colleagues would be in a position to talk with each other about the best ways to do our common work.
On another note, I could see educators voicing arguments similar to those of physicians with regard to why they don't need checklists (e.g. "teaching is an art," "I don't want to be restricted by a checklist," etc.).
Shortly after reading Bryce's comment I got a copy of an article on health care reform from my boss. (Thanks, Cid.) The piece, Making Health Care Better profiles Intermountain Health Care's efforts to improve health care and cut costs (go, Utah!)
IHC uses checklists (or protocols like them) but the improvement of health care system-wide comes not just from the checklists but from using and sharing data from them. IHC's computer system recommends certain treatments to doctors and nurses (the preferred treatments are developed by a team of doctors, nurses, and administrators from IHC). They are free to use those treatments or choose others. Those choices and the outcomes go back into the system. So, over time, IHC has developed a set of preferred practices, based on research and outcomes, and the evidence from those practices is visible.
Contrast this with what happens in education. Teachers prepare for class, using a combination of the things they have done in the past, innovations they prefer, ideas from colleagues, etc. My sense is that most teachers (and nearly all professors) spend a substantial amount of time in preparation, as well we should.
Then they hold class. There, they likely do most of the things they planned to do, leaving some out because of time constraints or because the class goes in a different direction. At some point students are supposed to show evidence that they learned. The professors evaluate them. Class ends. Students get final grades and go on to the next course.
The major difference between excellent health care and excellent college classes, then, isn't the preparation. It probably isn't the presence or absences of checklists either (though they cannot hurt, it seems). Instead it is transparency.
The transparency problem has two components. First, with the exception of student teachers and faculty undergoing review, no other qualified teacher ever sees what happens in a classroom. Second, no one ever sees the outcomes of what happens in a particular class. As a result, systemic improvement generally depends on training rather than results.
Why is the classroom so opaque? Tradition has something to do with it--universities still carry with them a model of master-disciple born in the middle ages, and that model suggests that the key relationship is between teacher and student, not between student and educational system. Academic freedom, at least as we have come to see it, influences as well.
But I think the main obstacles are not cultural, but systemic. After all, faculty favor collaboration with colleagues on nearly every other task (committees, research, writing, etc.) and most good teachers turn to colleagues for guidance all the time. The "fear" of having someone else watch and comment on your work is probably not too deep, particularly if that someone is a fellow faculty member.
But the system makes it impossible in three ways: first, almost no system of education believes in standard practices (a "meta-checklist" if you will). Teaching chemistry differs from teaching history; teaching first-year students isn't like teaching grad students. But why is this? Those few institutions with a campus-wide commitment to particular teaching approaches seem to have positive effects. Second, we schedule faculty work so nearly all professors are teaching simultaneously. Why don't colleagues visit your classes? Because they are in their own. Third, we don't know what outcomes would look like. What would be the result of a well-taught class in history? How would it differ from a similar course in, say, music?
I don't have good solutions to these transparency problems. But I can think of a simple first step. As part of campus faculty development efforts and accounted for in faculty workload, mandate (or incentivize) faculty members to enroll in one class taught by a colleague each year. A few benefits would be almost immediate. First, faculty development would be campus-wide rather than focused on the willing few. Second, faculty would learn from each other. Third, they would see the experience of students first-hand. And fourth, colleagues would be in a position to talk with each other about the best ways to do our common work.
Tuesday, January 5, 2010
Education checklists?
This morning NPR interviewed the surgeon Atul Gawande about his new book The Checklist Manifesto. In it he argues that surgeons do much better work when their surgeries are guided by checklists. As the surgery goes along, surgeons, nurses, and techs make sure they have covered all best practices by completing a checklist. Gawande reports that surgical errors declined by 35% when surgeons used a checklist. The checklists are based on those used by pilots and developed by Boeing.
I have often advised my students to create checklists, though more as a time management practice than as something that would make their performance better. But the Gawande interview left me wondering if colleges ought to use checklists much more frequently.
Take, for example, advising, mentoring, and other student support activities. Campuses recommend them for all students, but often the best students (or at least the most compliant) are those who take advantage, while at risk students don't. Do schools use mandatory checklists to ensure that all students take advantage of high impact experiences? (George Kuh and the NSSE people found that students of color and first-generation students are less likely than other students to participate in high impact activities like learning communities that are proven to improve student engagement.)
Would retention go up 35%?
Or consider faculty members. What if they went into every class with a checklist of best practices. Write outline on the board...Take roll...Elicit questions about previous class meeting...Uncover pre-existing understanding among students...Switch pedagogies every 15 minutes...Give students assignments to be completed in class...Check for understanding...
Would learning improve 35%?
If not, why? Is a failure to learn not the result of mistakes and overlooked opportunities? Are some students not retained because they simply "fall through the cracks"? Is education more complicated than surgery?
I have often advised my students to create checklists, though more as a time management practice than as something that would make their performance better. But the Gawande interview left me wondering if colleges ought to use checklists much more frequently.
Take, for example, advising, mentoring, and other student support activities. Campuses recommend them for all students, but often the best students (or at least the most compliant) are those who take advantage, while at risk students don't. Do schools use mandatory checklists to ensure that all students take advantage of high impact experiences? (George Kuh and the NSSE people found that students of color and first-generation students are less likely than other students to participate in high impact activities like learning communities that are proven to improve student engagement.)
Would retention go up 35%?
Or consider faculty members. What if they went into every class with a checklist of best practices. Write outline on the board...Take roll...Elicit questions about previous class meeting...Uncover pre-existing understanding among students...Switch pedagogies every 15 minutes...Give students assignments to be completed in class...Check for understanding...
Would learning improve 35%?
If not, why? Is a failure to learn not the result of mistakes and overlooked opportunities? Are some students not retained because they simply "fall through the cracks"? Is education more complicated than surgery?
Sunday, January 3, 2010
What do we make of the shakeout in higher ed open content?
First, a disclaimer. I am a neophyte in the opencourseware/open content world. But I've been following it for the past year, and am involved in Westminster's efforts to figure out how to play in that world. And if you follow this blog (all three of you :-)) then you will know it is central to some of my thinking about cost and quality in higher ed.
Having said that, it seems to me that over the past couple of months there have been some major changes in the higher ed open content landscape. First, Utah State University got rid of its opencourseware project, the second largest in the US. Second, MIT released a report on the costs of hosting its OCW project, with some ideas about how to sustain it in the future without the major institutional and grant funding that made it possible in headier economic times. Third, the Mellon Foundation consolidated one of its major grant programs that had funded a lot of open content work and its two grant officers left the foundation.
What do these three things imply?
1. They do not imply that open content or opencourseware are going out the window. There are still lots of efforts out there, and lots of successes to be noticed.
2. They do imply that individual institutions of higher education will struggle to provide open content infrastructure necessary for the "revolution" in learning that some people expect. (Check out this recent issue of the journal IRRODL for some of the revolution language.)
3. They suggest that the same economic downturn that makes open content attractive as an alternative to expensive higher education makes it difficult to provide that open content.
4. If (3) is correct, then future open content infrastructure will be more likely to be provided by for-profit companies (i.e. youtube.edu; ITunesU, etc.) or the federal government than by individual institutions of higher ed.
5. If individual institutions want to stay at the front, it seems that they ought to specialize. The future seems not to be MIT providing an entire university worth of courses online for free, but instead institutions leading in areas of their speciality. MIT in engineering, Utah State in agriculture, who (???) in liberal arts.
Having said that, it seems to me that over the past couple of months there have been some major changes in the higher ed open content landscape. First, Utah State University got rid of its opencourseware project, the second largest in the US. Second, MIT released a report on the costs of hosting its OCW project, with some ideas about how to sustain it in the future without the major institutional and grant funding that made it possible in headier economic times. Third, the Mellon Foundation consolidated one of its major grant programs that had funded a lot of open content work and its two grant officers left the foundation.
What do these three things imply?
1. They do not imply that open content or opencourseware are going out the window. There are still lots of efforts out there, and lots of successes to be noticed.
2. They do imply that individual institutions of higher education will struggle to provide open content infrastructure necessary for the "revolution" in learning that some people expect. (Check out this recent issue of the journal IRRODL for some of the revolution language.)
3. They suggest that the same economic downturn that makes open content attractive as an alternative to expensive higher education makes it difficult to provide that open content.
4. If (3) is correct, then future open content infrastructure will be more likely to be provided by for-profit companies (i.e. youtube.edu; ITunesU, etc.) or the federal government than by individual institutions of higher ed.
5. If individual institutions want to stay at the front, it seems that they ought to specialize. The future seems not to be MIT providing an entire university worth of courses online for free, but instead institutions leading in areas of their speciality. MIT in engineering, Utah State in agriculture, who (???) in liberal arts.
Subscribe to:
Posts (Atom)
