Wednesday, December 14, 2016

Ten Reflections from the Fall Semester

Notes from this semester. Each semester I jot down observations about organizational practices, usually inspired by events at my place of employment.  Every now and then I try to distill them into advice for myself. Most are obvious, once articulated, but they come to notice, usually, because things happen just the other way round.
  1. Always treat the people you work with as if they are smart; explain why you take a stand or make a decision in a manner that demonstrates that you know they are smart, critical, and open to persuasion by evidence and argument. Set high standards for yourself. Your institutional work should be at least as smart as your scholarly work.
    1. "it is better to be wrong than vague." - Stinchcombe
    2. If smart people are opposed to your idea, ask them to explain why. And listen non-dismissively and non-defensively. Remember, you goal is to get it right, not to get it your way.
  2. Do not put people in charge of cost cutting and budget reductions. Put them in charge of producing excellence within a budget constraint.

  3. Make sure everyone is able to say how many Xs one student leaving represents.  How much will it cost to do the thing that reduces the chance a student will get fed up with things?

  4. If most of what a consultant tells you is what you want to hear (or already believe), fire her.

  5. Don't build/design system and policies around worst cases, least cooperative colleagues, people who just don't get it, or individuals with extraordinarily hard luck situations. Do not let people who deal with "problem students" suggest or make rules/policy.

  6. Be wise about what you must/should put up for a vote and what you should not.  And if you don't know how a vote will turn out, they are are not prepared to put it up for a vote.  Do your homework, person by person.

  7. If a top reason for implementing a new academic program is because there's lots of interest among current students, pause. Those students are already at your school. What you want are new programs that are attractive to people who previously would not have given you a second look.

  8. If you are really surprised by the reaction folks have to an announcement or decision then just start your analysis with the realization that YOU screwed up.

    1. Related: and don't assume it was just about the messaging; you might actually be wrong and you should want to know whether that's the case.

  9. If you or someone else's first impulse when asked to get something done is to form a committee, put someone else in charge of getting that thing done.

  10. Train folks to realize that teams and committees in organizations are not representative democracies. The team does not want your opinions, feelings, experiences, or beliefs; it wants you enrich the team's knowledge base by reporting on a part of the world you know something about.  And that usually means going and finding out in a manner that is sensitive to your availability bias.  In the research phase, team members are the sense organs of the team. 



American Talent Initiative aims to recruit 50,000 highly qualified students from modest backgrounds

Well, this is good news. Unless, perhaps, you are already an institution that does this - sure the pool is a deep one, but what's the net effect when top schools skim the top of it? Still, attaching the research resources to the effort is a good thing - way too much seat-of-the-pants policy and practice in this area.

Looking for Low Income Students

A group of 30 top colleges and universities wants to enroll more low-income students, but critics question whether the focus should be elsewhere.
By Rick Seltzer Inside Higher Ed December 13, 2016

A new effort to enroll low- and moderate-income undergraduates at colleges and universities with high graduation rates is being announced today in an attempt to have more students from modest backgrounds graduate from prestigious campuses seen as opening doors to top careers.

The effort, called the American Talent Initiative, aims to add 50,000 highly qualified students from modest backgrounds to campuses with high graduation rates by the year 2025. A group of 30 colleges and universities have signed on to the initiative, which is being coordinated by the nonprofit Aspen Institute’s College Excellence Program and Ithaka S+R. Bloomberg Philanthropies is providing $1.7 million over two years to start the project, money that won’t go directly to colleges and universities but will be used to fund research on their efforts and related activities.

Read more at Inside Higher Ed

Thursday, December 1, 2016

The "Core" COULD actually be a core

In the Chronicle of Higher Education Nicholas Lemann argues for an alternative approach to a core curriculum that is explicitly focused on intellectual skills and METHODS. The core courses he proposes would all be interesting to teach:
  • Information Acquisition: kinds, acquiring, evaluating
  • Cause and Effect: science as style of thought
  • Interpretation: close reading of texts
  • Numeracy: quantity in everyday life
  • Perspective: the limits of one's own viewpoint
  • Language of Form: intelligently seeing/producing visual information
  • Thinking in Time: thinking historically
  • Argument: how to make a compelling and analytically sound argument
One element of what Lemann is responding to should sound familiar: "Quite a few colleges … devising a new undergraduate liberal-arts curriculum … these new curricula often identify a suite of intellectual skills … [but] permit a wide array of existing courses to fulfill the requirements … [thus] declaring victory simply by pasting on a new label."

Or, he continues:
Or they define the new requirements in terms of "learning outcomes" rather than course content, which puts the emphasis on devising an end-of-course assessment rather than on designing the course itself. Or they offer courses on broad interdisciplinary subjects, with words like "ethics," "values," or "justice" in their titles, rather than on the inescapably different project of identifying fundamental methods of understanding and analysis.
And the result of that is something my own school has: a core curriculum that is neither core nor curriculum.

More to the point, many schools (my own included) allow even a "core" which is called skills or competency based to be captured by colleagues who want the content - especially values and worldviews - that they champion to be required for all and who use core requirements to drive enrollments in their departmental courses. The "core" becomes a symbolic expression of whose intellectual and ideological commitments are on top at the moment and then a whole bunch of organizational ritual and hoohah emerges to regularly remind all of whose game it is and to channel resources in their direction. Until the next reimagining of the core elevates some other group.

My colleagues can read the article here.  If you have premium access to the Chronicle, you can read the whole article there.

The Case for a New Kind of Core

NOVEMBER 27, 2016 

When I was a professional-school dean (at Columbia University’s Graduate School of Journalism), we had no choice but to try to define the specific content of an education in our field. The premise was that if you want to practice a profession, there is a body of material you must master, at least in the early part of your education. That perspective led me to urge, this year in The Chronicle Reviewthat undergraduate colleges move in a similar direction: a core curriculum.

READ MORE at CHE
-->

Wednesday, September 28, 2016

College Affordability Expert on the Daily Show

A friend and co-author has a new book and did an interview on The Daily Show last night. You don't see too many sociologists on TV, BTW. Sara's new book is a research-based look at the challenges of paying for higher education, with solutions.


Paying the Price: College Costs, Financial Aid, and the Betrayal of the American Dream




Thursday, September 22, 2016

"But even if they are not valid, they do tell you something...."

Remember, "validity" means "they measure what you think they measure." "Data driven" can also mean driven right off the side of the road.

From Inside Higher Ed

Zero Correlation Between Evaluations and Learning

New study adds to evidence that student reviews of professors have limited validity.
September 21, 2016
A number of studies suggest that student evaluations of teaching are unreliable due to various kinds of biases against instructors. (Here’s one addressing gender.) Yet conventional wisdom remains that students learn best from highly rated instructors; tenure cases have even hinged on it.
What if the data backing up conventional wisdom were off? A new study suggests that past analyses linking student achievement to high student teaching evaluation ratings are flawed, a mere “artifact of small sample sized studies and publication bias.”
“Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between [student evaluations of teaching, or SET] ratings and learning,” reads the study, in press with Studies in Educational Evaluation. “Our up-to-date meta-analysis of all multi-section studies revealed no significant correlations between [evaluation] ratings and learning.”

Sunday, August 14, 2016

House of Cards

A Facebook post called my attention to a neat little article about why swimming rules only recognize hundredths of seconds even though modern timing technology allows much more precise measurements. The gist is this: swimming rules recognize that construction technology limits the precision with which pools can be built to something like a few centimeters in a 50 meter long pool.  At top speed a swimmer moves about 2 millimeters in a thousandth of a second.  So, if you award places based on differences of thousandths of a second, you can't know if you are rewarding faster swimming or the luck of swimming in a shorter lane.

This observation points to the more general phenomena of false precision, misplaced concreteness (aka reification, hypostatization), and organizational irrationality rooted in sloppy and abusive quantification.

These are endemic in higher education.

Students graduate with a GPA and it's taken as a real, meaningful thing. But if you look at what goes into it (exams designed less and more well, subjective letter grades on essays, variable "points off" for rule infractions, quirky weighting of assignments, arbitrary conversions of points to letter grades, curves, etc.), you'd have to allow for error bars the size of a city block.

Instructors fret about average scores on teaching evaluations.

"Data driven" policies are built around the analysis of tiny-N samples that are neither random nor representative.

Courses are fielded or not and faculty lines granted or not based on enrollment numbers with no awareness of the contribution of class scheduling, requirement finagling, course content overlap, perceptions of ease, and the wording of titles.

Budgets are built around seat-of-the-pants estimates and negotiated targets.

One could go on.

The bottom line is that decision makers need to recognize how all of these shaky numbers are aggregated to produce what they think are facts about the institution and its environment.  This suggests two imperatives. First, we should reduce individual cases of crap quantification.  Second, when we bring "facts" together (e.g., enrollment estimates and cost of instruction) we should adopt an "error bar" sensibility - in it's simplest form, treat any number as being "likely between X and Y" - so that each next step is attended by an appropriate amount of uncertainty rather than an inappropriate amount of fantasized certainty.

Monday, August 1, 2016

"Free College" and the System of Higher Education




Finally, someone is writing about the consequences of "free" college for the system of higher education in the US.


From The Chronicle of Higher Education 1 August 2016

How Clinton’s ‘Free College’ Could Cause a Host of Problems

By SCOTT CARLSON
AND BECKIE SUPIANO

The policy proposals of presidential campaigns aren’t often burdened by details or even realism. A candidate’s ideas are supposed to represent vision, ambitions, principles — all while taking on the latest American anxiety.

These days, some of that anxiety concerns the cost of college and the notion that student debt burdens young people as they head out to get jobs, buy homes, and start families. Hillary Clinton’s answer is her “New College Compact,” which includes a plan — adapted from her tenacious primary opponent, Sen. Bernie Sanders — that would cover tuition for students from families earning up to $125,000 a year.

“College used to be pretty affordable,” says a fact sheet on Mrs. Clinton’s compact. “For millions of Americans, that’s not the case anymore.” Colleges’ systems of grants and other financial assistance are complicated, and “free tuition” is a lot easier to pitch than a plan to tweak the existing patchwork of aid. Simple messages tend to resonate best.

And this message is a particularly resonant one. Higher education is widely seen as a necessary step on the road to a middle-class lifestyle, and most policy makers agree that the country needs a more educated work force. But as more of the burden of paying for college shifts to students and their families, proposals like Mrs. Clinton’s make a powerful suggestion: that higher education is a public good, which deserves to be treated as such.

The plan is grand — and very likely dead on arrival in Washington. Although the notion of free college is popular among progressives and young people, conservatives — who will probably retain control of the House of Representatives and many state governments after November — have balked at the cost of various free-college plans. Even some left-leaning policy wonks have questioned whether the plan would drive up tuition, put new burdens on the tax system, or even undermine college access.

Let’s set aside for a moment the question of whether the plan could ever become reality and treat it as a thought experiment: If Mrs. Clinton’s plan passed, what would happen to the higher-ed landscape? Many of the specifics aren’t known yet. But one thing is clear: Policy makers could write a free-college plan that does significant harm and questionable good.

PRIVATE COLLEGES IN PERIL

First in line for harm, most experts agree, would be private colleges. Although many people (and some policy makers) picture elite, wealthy institutions at the mention of “private colleges,” the category also includes hundreds of small, remote institutions, with tiny endowments.
“These colleges are concentrated in rural areas in the Midwest and Northeast, where high-school populations have been fairly stagnant,” says Robert Kelchen, an assistant professor of higher education at Seton Hall University. What’s more, he says, high-school graduates are increasingly minority or first-generation college students with lower incomes. “Because of that, these students might be more price-sensitive and may be interested in going to a public college rather than a private college.”

There’s a big variable here: Mrs. Clinton’s free-college plan does not make clear whether students at private colleges could still get grants and loans from the federal government. And while free tuition would surely appeal to many families, students don’t choose colleges on price alone. They also care about finding a strong academic program and a good fit. Geography, too, is key: Most students go to college relatively close to home.

But if public colleges became free for those lower-income students, says Kent John Chabotar, a former president of Guilford College, “small private colleges without endowments in states with highly regarded public universities — particularly the flagship universities — would be in trouble.”

The private colleges would have to compete to attract students who would be less prepared for college and have lower expected family contributions. “You’re going to see a combination of dropping enrollments and skyrocketing tuition discounting, killing off the weaker, private, unendowed colleges,” Mr. Chabotar says.


PUBLIC COLLEGES PRESSURED

So let’s say that the migration happens, and a new crop of students chooses public institutions over the private ones. It’s unclear that regional public and community colleges have enough capacity to meet that demand.

Public two- and four-year colleges already enroll more than three-quarters of the nation’s undergraduates. Even if a college had been planning to grow when Mrs. Clinton’s policy took effect, government funding probably would not keep pace with its needs over time, says Donald Hossler, a senior scholar at the Center for Enrollment, Research, Policy & Practice at the University of Southern California.

Colleges, he says, would be expected to educate more people with fewer resources per student. The quality of public education could erode. When enrollment is high and funding is tight, it can be hard for students to get all the classes they need to graduate on time.

At flagships and other selective public colleges, the picture would be more complicated. Flagships already tend to enroll more relatively affluent students, whose socioeconomic advantages give them an edge in admissions. Unless the government were to give the flagships some incentive to grow, they’d have little reason to take on more students. That would mean even more competition for a fixed number of seats.

So while free in-state tuition might sound like a boon to low-income students, it doesn’t help them much if they can’t get into the public college they want to attend, says Donald E. Heller, provost and vice president for academic affairs at the University of San Francisco.

In fact, some experts worry that free tuition for most families could exacerbate existing inequalities and further stratify higher education. While poor students would attend crowded, lower-tier public colleges at no cost, affluent students could buy their way into elite colleges, public or private.

Flagships have long worked to bring in more revenue from sources beyond state appropriations, like tuition — by enrolling more out-ofstate students, for instance. That’s unlikely to change. One big question is how much flexibility the institutions would retain in those efforts. What would students whose families make $125,000 or more be asked to pay?

If the policy applies to out-ofstate students, that eliminates a source of additional revenue. But if it applies only to in-state students, enrolling out-of-staters with family incomes below $125,000 would get harder when those students could attend their in-state colleges free, says Robert K. Toutkoushian, a professor in the Institute of Higher Education at the University of Georgia.


BETTER STUDENT OUTCOMES?

Free in-state tuition might also change when some students enroll. Mrs. Clinton has proposed that the program start out covering families making $85,000 or less, with the cap rising $10,000 annually for the next four years, until all families making less than $125,000 are covered. A family making $104,000 in the first year of the program might hold off on sending their children to college for a couple of years, Mr. McPherson says.

You might think that a plan that saves students money, possibly reducing how much they must work outside of class, ought to help them graduate, Mr. Hillman says. But graduation rates are higher at private four-year colleges than at public ones. That probably can’t be chalked up entirely to the colleges themselves — the students who enroll matter, too — but it makes it harder to think of the plan as a boon to college completion.
In the end, the free-college proposal is about one thing: mitigating debt. “Every student should have the option to graduate from a public college or university in their state without taking on any student debt,” says Mrs. Clinton’s website.

Sure, students from families making up to $125,000 wouldn’t have to borrow for tuition, but that doesn’t mean they wouldn’t have to borrow. They would still have to pay their living expenses, which can be a bigger burden than tuition, especially for needy students. Studies have shown that students on a low hourly wage have a hard time covering those bills.

Barring sizable government investment, many students would still take out loans, a pattern already established in other countries that have tried “free college.” Even at the handful of wealthy American colleges that meet students’ full financial need — accounting for the full cost of attendance, without loans — some students still borrow.

Here’s one more unanswered question: Does “free” mean tuition alone, or does it include fees? That’s no small detail: If colleges can’t get more tuition out of most students, they might look to increase fees instead.


ECONOMIC ENGINES

Colleges are economic engines in their towns — machines that move money around, particularly in rural communities. In many parts of the Northeast, Rust Belt, Midwest, and beyond, small colleges are anchor institutions, helping to prop up communities that long ago lost the manufacturers and farmers that helped create them in the first place.

Let’s assume that students chase free tuition at the public colleges, abandoning fragile private colleges and leading to their closure. What would happen to a place like Rensselaer, Ind., home of Saint Joseph’s College?

Saint Joseph’s is a Roman Catholic institution with 2,000 students; 45 percent are first-generation students, most of whom would be covered by the Clinton plan. “If you take 45 percent of our population, and you allow them to go to Purdue or Indiana University or any of the state schools in Indiana for free, more than likely they are not going to be coming here,” says Robert A. Pastoor, the college’s president. “The viability of the institution is going to be seriously called into question.” Indiana has 31 private institutions, he adds, and many of them would find themselves in the same situation.

In a town of 6,000, the college employs about 250 people. and is a significant economic engine. Students, parents, and alumni shop at the grocery store, eat at the restaurants, sleep in the hotels. Locals go to sports games, celebrate Mass in the college’s Romanesque chapel, and hold wedding receptions and meetings in college facilities.

“All of that would go away,” says Mr. Pastoor, “and there is nothing to take its place.”

Wednesday, January 13, 2016

Student Evaluations of Teaching: WHY is this still a thing?


My institution just created a data science major. But it doesn't care about using data in honest and robust ways any more than other institutions.

It's gotten to the point that it's intellectually embarrassing and ethically troubling that we are still using student evaluations of teaching (SET) in their current form for assessing instructor job performance. It is laughable that we do so with numbers computed to two decimal places. It is scandalous that we ignore the documented biases (most especially gender-based). But we do.

Why isn't this an active conversation between faculty and administrators?  I certainly find teaching evaluations helpful - trying to understand why I got a 3.91 on course organization but a 4.32 on inspiring interest is a useful meditation on my teaching practice.  I have to remind myself that the numbers themselves do not mean much.

Telling me where my numbers stand vis a vis my colleagues or the college as a whole FEELS useful and informative, but is it? I THINK I must be doing a better job than a colleague who has scores in the 2.0 - 3.0 range. But doing a better job at what? If you think hard about it, all you can probably take the bank is that I am better at getting more people to say "Excellent" in response to a particular question. The connection between THAT student behavior and the quality of my work is a loose one.

Maybe I am on solid ground when I compare my course organization score to my inspires interest score. MAYBE I am on solid ground when I compare my course organization score in one class to the same score in another the same semester or the same class in another year. I might, for example, think about changes I could make in how I organize a course and then see if that score moves next semester.

But getting seduced by the second decimal place is ludicrous and mad. Even fetishizing the first decimal place is folly. For that matter, even treating this as an average to begin with is bogus.

If you also use these numbers to decide whether to promote me, you've gone off into the twilight zone where the presence of numbers gives the illusion of facticity and objectivity. Might as well utter some incantations while you are at it.

Some new research adds another piece of evidence to the claim that the validity of the numbers in student evaluations of teachers is probably pretty low. Validity means "do they measure what you think they measure?" The answer here is that they do not. Instead, they measure things like "what gender is your instructor?" and "what kind of grade do you expect in this course?"

These researchers even found gender differences in objective practices like "how promptly were assignments graded" and these persisted when the students were misinformed about gender of instructors.

Let's start implementing a policy we can have some respect for. No more averaging. No more use of numerical scores in personnel review. No more batteries of questions that ask more or less the same thing (thus distorting the positivity or negativity of the overall impression).

As John Oliver asks, "why is this still a thing?"

Friday, January 1, 2016

NPR's Cladio Sanchez' "6 Education Stories To Watch In 2016"

NPR's senior education correspondent offers his predictions for stories in education in 2016.


1. The New Federal Education Law

The long, grueling fight to overhaul the 14-year-old No Child Left Behind law is over, but that'll turn out to be the easy part. The new Every Student Succeeds Act returns most government oversight of schools back to states. But there are no guarantees that the states will do a better job than the federal government in two key areas: closing the achievement gap and raising the performance of the absolute worst schools.

There will be some relief for students burdened by excessive testing. But for the most part states will continue to rely on test scores, using them to punish schools rather than for improving curriculum and instruction. Reading and math scores will drop for all kids on the new, tougher standardized tests linked to the Common Core. But the dismal performance of groups that struggle will trigger more scrutiny from civil rights groups in 2016. We'll also see those groups pressure states to deal with teacher quality and funding.

2. Moving On From Common Core

The controversy over the much-maligned Common Core State Standards will diminish. States will continue their efforts to re-brand or rename the standards, while for the most part following them. Despite the political controversy, the push for high academic standards will continue, and we'll see little of the "race to the bottom" that happened under NCLB.

3. Charter Schools Under A Microscope

The charter school movement will celebrate its 25th anniversary in 2016. With 6,700 schools and nearly 3million students across 43 states and the District of Columbia, charters are a powerful force. The federal government has poured billions of dollars into charters, and polling shows that a majority of Americans support them. But you can expect these publicly funded, privately run schools to face new scrutiny, and new criticism.

We'll see more scandals involving fraud, corruption and mismanagement, despite efforts to weed out "bad actors" who've exploited weak charter laws in several states. As Joe Nathan, a senior fellow at the Center for School Change, who helped write charter school legislation in 32 states, puts it: "We have not done enough to deal with the crooks and charlatans, of which we have our share."

Charters will also be one of the very few education issues to get any attention in the presidential campaign.

4. Dreamers Dreams Deferred

There will be an even stronger backlash against the push for greater access to college for undocumented students. Dreamers — students brought to the U.S. illegally as children — will face greater opposition because of the stalemate over immigration reform. The angry, anti-immigrant rhetoric from Republicans running for president will also shape this debate. Look for state lawmakers to consider even tougher measures to deny dreamers any benefits and push them deeper into a legal and educational limbo.

5. Goodbye Race-Conscious Admissions

Watch for the U.S. Supreme Court to ban race in college admissions, forcing institutions to abandon affirmative action policies. Schools will have to rethink how they recruit and enroll students in efforts to increase diversity. This will fuel an already tense situation on many campuses. Expect minority student protests and campus unrest to intensify.

6. Student Debt Takes Center Stage

Higher education leaders, or what presidential candidate Senator Marco Rubio calls "the higher-ed cartel," effectively killed the Obama administration's attempt to create a more transparent, consumer-friendly way for students and parents to rate colleges. But with many of the presidential candidates calling for tuition-free or debt-free college, we'll see these institutions undertake a more serious discussion about changing their pricing policies — largely out of fear that lawmakers in Washington will step in and do it for them.


Last Year's List


  1. Standardized Testing Under Fire
  2. More Troubles For The Common Core
  3. In Congress, Deeper Divisions
  4. Focus On Campus Behavior
  5. Teacher Evaluation, Training, And The Vergara Fallout
  6. The Ferguson Effect: New Scrutiny For School Police


And some from NPR's "crowd sourced" predictions for 2015.


  1. Blended Learning As A Daily Practice
  2. More Scrutiny of Student Data
  3. Broader Disclosure On Student Loan Defaults
  4. Moving On From Common Core Debates
  5. Reauthorization of No Child Left Behind; More School Choice
  6. Customizable, Game-Like Platforms
  7. Transition For The Online Education Space. "Snackable" learning will become a large part of the online education menu.
  8. More Options For Student Borrowers
  9. Competency-Based Education Picks Up
  10. More Nuanced Kinds Of Data In Schools
  11. The Digital Classroom Meets Labor Issues