Wednesday, January 13, 2016

Student Evaluations of Teaching: WHY is this still a thing?


My institution just created a data science major. But it doesn't care about using data in honest and robust ways any more than other institutions.

It's gotten to the point that it's intellectually embarrassing and ethically troubling that we are still using student evaluations of teaching (SET) in their current form for assessing instructor job performance. It is laughable that we do so with numbers computed to two decimal places. It is scandalous that we ignore the documented biases (most especially gender-based). But we do.

Why isn't this an active conversation between faculty and administrators?  I certainly find teaching evaluations helpful - trying to understand why I got a 3.91 on course organization but a 4.32 on inspiring interest is a useful meditation on my teaching practice.  I have to remind myself that the numbers themselves do not mean much.

Telling me where my numbers stand vis a vis my colleagues or the college as a whole FEELS useful and informative, but is it? I THINK I must be doing a better job than a colleague who has scores in the 2.0 - 3.0 range. But doing a better job at what? If you think hard about it, all you can probably take the bank is that I am better at getting more people to say "Excellent" in response to a particular question. The connection between THAT student behavior and the quality of my work is a loose one.

Maybe I am on solid ground when I compare my course organization score to my inspires interest score. MAYBE I am on solid ground when I compare my course organization score in one class to the same score in another the same semester or the same class in another year. I might, for example, think about changes I could make in how I organize a course and then see if that score moves next semester.

But getting seduced by the second decimal place is ludicrous and mad. Even fetishizing the first decimal place is folly. For that matter, even treating this as an average to begin with is bogus.

If you also use these numbers to decide whether to promote me, you've gone off into the twilight zone where the presence of numbers gives the illusion of facticity and objectivity. Might as well utter some incantations while you are at it.

Some new research adds another piece of evidence to the claim that the validity of the numbers in student evaluations of teachers is probably pretty low. Validity means "do they measure what you think they measure?" The answer here is that they do not. Instead, they measure things like "what gender is your instructor?" and "what kind of grade do you expect in this course?"

These researchers even found gender differences in objective practices like "how promptly were assignments graded" and these persisted when the students were misinformed about gender of instructors.

Let's start implementing a policy we can have some respect for. No more averaging. No more use of numerical scores in personnel review. No more batteries of questions that ask more or less the same thing (thus distorting the positivity or negativity of the overall impression).

As John Oliver asks, "why is this still a thing?"

Friday, January 1, 2016

NPR's Cladio Sanchez' "6 Education Stories To Watch In 2016"

NPR's senior education correspondent offers his predictions for stories in education in 2016.


1. The New Federal Education Law

The long, grueling fight to overhaul the 14-year-old No Child Left Behind law is over, but that'll turn out to be the easy part. The new Every Student Succeeds Act returns most government oversight of schools back to states. But there are no guarantees that the states will do a better job than the federal government in two key areas: closing the achievement gap and raising the performance of the absolute worst schools.

There will be some relief for students burdened by excessive testing. But for the most part states will continue to rely on test scores, using them to punish schools rather than for improving curriculum and instruction. Reading and math scores will drop for all kids on the new, tougher standardized tests linked to the Common Core. But the dismal performance of groups that struggle will trigger more scrutiny from civil rights groups in 2016. We'll also see those groups pressure states to deal with teacher quality and funding.

2. Moving On From Common Core

The controversy over the much-maligned Common Core State Standards will diminish. States will continue their efforts to re-brand or rename the standards, while for the most part following them. Despite the political controversy, the push for high academic standards will continue, and we'll see little of the "race to the bottom" that happened under NCLB.

3. Charter Schools Under A Microscope

The charter school movement will celebrate its 25th anniversary in 2016. With 6,700 schools and nearly 3million students across 43 states and the District of Columbia, charters are a powerful force. The federal government has poured billions of dollars into charters, and polling shows that a majority of Americans support them. But you can expect these publicly funded, privately run schools to face new scrutiny, and new criticism.

We'll see more scandals involving fraud, corruption and mismanagement, despite efforts to weed out "bad actors" who've exploited weak charter laws in several states. As Joe Nathan, a senior fellow at the Center for School Change, who helped write charter school legislation in 32 states, puts it: "We have not done enough to deal with the crooks and charlatans, of which we have our share."

Charters will also be one of the very few education issues to get any attention in the presidential campaign.

4. Dreamers Dreams Deferred

There will be an even stronger backlash against the push for greater access to college for undocumented students. Dreamers — students brought to the U.S. illegally as children — will face greater opposition because of the stalemate over immigration reform. The angry, anti-immigrant rhetoric from Republicans running for president will also shape this debate. Look for state lawmakers to consider even tougher measures to deny dreamers any benefits and push them deeper into a legal and educational limbo.

5. Goodbye Race-Conscious Admissions

Watch for the U.S. Supreme Court to ban race in college admissions, forcing institutions to abandon affirmative action policies. Schools will have to rethink how they recruit and enroll students in efforts to increase diversity. This will fuel an already tense situation on many campuses. Expect minority student protests and campus unrest to intensify.

6. Student Debt Takes Center Stage

Higher education leaders, or what presidential candidate Senator Marco Rubio calls "the higher-ed cartel," effectively killed the Obama administration's attempt to create a more transparent, consumer-friendly way for students and parents to rate colleges. But with many of the presidential candidates calling for tuition-free or debt-free college, we'll see these institutions undertake a more serious discussion about changing their pricing policies — largely out of fear that lawmakers in Washington will step in and do it for them.


Last Year's List


  1. Standardized Testing Under Fire
  2. More Troubles For The Common Core
  3. In Congress, Deeper Divisions
  4. Focus On Campus Behavior
  5. Teacher Evaluation, Training, And The Vergara Fallout
  6. The Ferguson Effect: New Scrutiny For School Police


And some from NPR's "crowd sourced" predictions for 2015.


  1. Blended Learning As A Daily Practice
  2. More Scrutiny of Student Data
  3. Broader Disclosure On Student Loan Defaults
  4. Moving On From Common Core Debates
  5. Reauthorization of No Child Left Behind; More School Choice
  6. Customizable, Game-Like Platforms
  7. Transition For The Online Education Space. "Snackable" learning will become a large part of the online education menu.
  8. More Options For Student Borrowers
  9. Competency-Based Education Picks Up
  10. More Nuanced Kinds Of Data In Schools
  11. The Digital Classroom Meets Labor Issues