Data Mining Ethics at the RMAIR Conference

I had the pleasure of presenting my paper on ethics and data mining at the Rocky Mountain Association for Institutional Research Conference today. First off, my thanks go out to the conference organizers for putting on an excellent conference. And then my thanks go to all of the people who had kind words and/or challenging questions about it.

The paper looks at the ethical side of a growing force in institutional research and higher education management. Data mining and predictive analytics are increasingly used in higher education to classify students and predict student behavior. But while the potential benefits of such techniques are significant, realizing them presents a range of ethical and social challenges. The immediate challenge considers the extent to which data mining’s outcomes are themselves ethical with respect to both individuals and institutions. A deep challenge, not readily apparent to institutional researchers or administrators, considers the implications of uncritical understanding of the scientific basis of data mining. These challenges can be met by understanding data mining as part of a value-laden nexus of problems, models, and interventions; by protecting the contextual integrity of information flows; and by ensuring both the scientific and normative validity of data mining applications.

I'll be posting highlights of the paper in blog-sized chunks over the next week or two. For those who can't wait, the full paper is posted at SSRN with the rest of my papers, and the PowerPoint presentation is available through Google Docs (update: it turns out Google Drive doesn't support animations) YouTube If I get really ambitious I'll record the narrations from to the slides and you can get, essentially, the whole presentation.

Getting Students to Understand Workload

Last week, Jill Rooney published an excellent post on questions students should never ask. She duly noted the general principle that there is no such thing as a stupid question, a principle that, having worked at Disneyland while an undergraduate, I know to be false: "What time is the nine o'clock parade?" is a stupid question. The rest, however, is excellent, mainly because the questions boil down to questions that students should ask themselves.

Today she followed that up with an equally useful post on questions students should ask. It includes a question that I pose to all of my students at the beginning of the semester:

@jillrooney2 Single best question: do they have 135 hours to put into this course over the next semester? So few understand that.
— Jeffrey Alan Johnson (@the_other_jeff) August 27, 2012

Students frequently don't quite get the time required for a course; the refrain of "this isn't the only class I'm taking" is all too common. To help them understand that, I work out a detailed time budget for the course, using the three-to-one rule of thumb. That's 135 hours total for a standard three-credit course at my institution. For my Introduction to Comparative Politics course this fall, the budget looks like this:

	Number	Time Each	Total Time
Class Meetings	28	1.5	42
Final Exam Meeting	1	2	2
Chapters	9	1	9
Additional Readings	9	0.5	4.5
Lynch Reading	1	12	12
Current Events	16	1	16
Quizzes	9	0.5	4.5
Exams	2	2	4
Country Study	1	20	20
Group Paper	1	20	20
Office Hours	2	0.5	1

That forms the basis for a section on my syllabus:

Workload

According to the accreditation standards that validate your degree as a legitimate one, to receive three semester credit hours requires 135 hours of study, including not more than 45 hours in class. In this course, study hours are budgeted as follows: Class meetings (44 hours), Readings (41.5 hours), Assignments (48.5 hours), Office Hour Consultations (1 hour).

When I go over the syllabus, I make clear that 135 hours is a non-negotiable standard, and students should expect to do poorly if they can't put in the required time. If their time commitments don't permit that, I suggest that they should consider cutting back on either their commitments or their expectations for performance.

Since I started doing that (back when I was a full-time faculty member rather than a bean counter—but they're such glorious beans that tell us so much!) I found that  I receive fewer complaints about the workload, and when I did they were specific things that I needed to respond to, e.g., "We can't get this assignment done in the time you think it will take us." That's especially good to me, as I tend to think that most students who complain about seemingly trivial things usually either have a legitimate concern but can't articulate it in relation to legitimate standards, or they have unrealistic expectations. This solves both of those concerns.

Ted Striker, African Gods, and Charles Peirce.

A friend reminded me of this, from my dissertation. I thought I'd share.

The problem at the heart of The Gods Must be Crazy is that the interpretants of the pilot and the Bushman are so fundamentally different that the bottle ceases to exist in a way that is remotely recognizable to someone from, literally in this case, a “Coca-cola” culture.  How, then, can the interpretant be real?  In fact, it is the reality of interpretant that creates the situation in the first place.  The fall of the bottle from the airplane demonstrates this.  We can see several interpretants at work for the pilot: soft drinks, garbage, the emptiness of the Kalahari.  If these are different—if the soft drink is not fully consumed, if the pilot thinks in terms of recycling rather than waste, if the pilot recognizes that the Kalahari is not so empty that the bottle could not possible hit anyone—the bottle is no longer something to be thrown out of the window.

The same is true for the Bushman.  In this case, we see the gods, the various activities for which the bottle can be used, communal order, and the mythological geography of the clan creating the bottle as it is used.  The Bushman’s belief in the existence of anthropomorphic gods who live in the sky leads him to believe that anything falling from the sky must have been given to him by the gods.  The belief that these gods have human-like emotions and motivations leads first to the sense that the bottle is a gift when it is brought into relationship with the everyday activities and then that it is a trick when it upsets social order within the clan.  The clan’s mythological geography—the belief in the “end of the Earth”—sparks the odyssey to permanently dispose of the bottle.  Like sign and object, interpretant displays its reality causally.  It causes something to be different than it might have been.  In so doing, it links the sign-object to other signs and objects.

But interpretant does more than this; it also links sign and object to each other.  When we ask the meaning of the sign “bottle” or “television” we can answer in two ways.  One is with description.  We explain what a bottle looks like, feels like, sounds like, and perhaps tastes and smells like.  We thus make reference to object.  Similarly, when we ask what an object is, we also answer in two ways that correspond to the question of a sign.  Analogous to the descriptive reference to the object is the nominative reference to sign: that object is a bottle or a television.  But neither of these ways fully satisfies us, much like the running joke in the film Airplane!:

“I have a message from headquarters.”
“Headquarters?  What is it?”
“It’s a big building where generals meet, but that’s not important right now.”

The joke is repeated several times: getting to a hospital (“a big building with patients in it”), helping out in the cockpit (“the small room at the front of the plane where the pilots sit”).  It is funny because the response doesn’t answer what the character is asking about—what any person would mean when they ask the question.  The character wants to know about the practices that relate to the sign, not about the object to which the sign corresponds.  They want to know not what objects the signs “headquarters,” “hospital,” and “cockpit” refer to, but what practices went on in these places that are relevant at the moment.  The character is asking about Thirdness.  Ted Striker wants to know what happened at headquarters, or what kind of help is needed in the cockpit; Captain Oveur wants to know why the passengers need to be brought to the hospital.  We can thus describe both sign and object in terms of a common interpretant, which unifies the sign and object.

This is the key function of interpretant.  In unifying sign and object through practice, interpretant unifies the natural and social aspects of reality in a single entity.  Peirceian semiotics, while idealist in that it posits the reality of ideal constructs, does not simply degenerate into an anything goes philosophy, one where the content of one’s own mind has no bounds but those of imagination, and in which unicorns and wormholes can be said to exist in exactly the same sense as a the computer on which I am typing.   The ideal is inexorably linked to the material in the sign-object-interpretant relationship.  Yet Peirce also shows that there can be no object without a sign, even if the sign is “that thing we just found.”  The reality of the object is linked just as inexorably to sign through interpretant.   Interpretant—which we should think of as sign and object in practice—thus serves to link the natural (object) and the social (the signs that we create for these objects) into a single fabric of reality.

Using Neutral Values in Survey Questions

What, pray tell, does a "neutral" response to a survey item tell institutional researchers?

The issue arose in an interesting discussion of survey methodology on the AIR LinkedIn group today. Erin Aselas was crowdsourcing solutions to response options on forced-choice Likert items, the preference of her university's administration. This being the Internet, Erin's question gave way to a broader discussion of whether the forced choice was a good idea or not. The discussion hinged on the question of whether, when asking someone to agree or disagree with a statement, "neutral" was the same as "no opinion."

Aside: Yes, I know my social media credibility just evaporated by mentioning that I use LinkedIn. That's where my colleagues happen to be, and the only thing that makes social media valuable is the people you connect to, so I'll live with it. These are good people.

Why the AIR Nominees Should Join Twitter

I voted today in the officer elections for the Association for Institutional Research. AIR is the main professional organization in North America for the people who do reporting, business intelligence, and assessment for higher education—my professional colleagues. Like most such organizations it relies on its members to serve as officers, board members, and committee members to function. The slate of candidates seems quite capable. I don't know any of them personally, though I've connected with one virtually (more on that below). Certainly, I am grateful that they've volunteered their time to make a great organization work.

I have been a member of AIR for about a year, so I hardly know the players in institutional research. But as someone who is active in the AIR LinkedIn group and who follows a fair number of institutional researchers on Twitter, I expected to recognize at least some of the candidates. With the exception of Ellen Peters, who is also involved in the LinkedIn group, social media users are absent. So far as I can tell, none use Twitter. Not one of the candidate's statements mentioned AIR's social media use. And that's a problem.

How Many More Syrians?

The deaths of three journalists are little in the grand narrative of somewhere between 3,000 and 7,500 lives lost in the Syrian uprising. But I had followed Marie Colvin's reports from Homs, saw Remi Ochlik's photographs of Libya and stayed up at night watching Rami al-Sayed's live video of Bama Amr. They, and others as well who have had the courage to tell Syria's story, made the conflict human for me. And in doing so, it made them human as well. So today I feel their loss.

It feels so trivial to write a letter to my Congressmen about this. I was a Marine; I feel like I should join the fight somehow. But this simple act is what the people of Syria are dying for: the right to say to the government that it should change its course. It isn't trivial for them.

I sent the following to my Representative and Senators today:

The Real "Bill Gates" Rules

So I’ve grown tired of the endlessly circulating "Bill Gates' Rules for Life" graduation speech that never actually happened. Had any of the last generation of business, technology, or intellectual leaders followed them we would still be using paper ledgers to build Ford Pintos. These rules are about as far from what we should be teaching students as one can get.

Let's think about what the rules would really be in today's economy.

Rule 1: Life isn’t fair. The people who want you to get used to it usually benefit from it at your expense. Don’t get used to it; do something about it. 

Rule 2: The world doesn’t care about your self-esteem. But if you wait until you accomplish something to feel good about yourself, you’ll never accomplish anything. 

Rule 3: Whether you make $60,000 a year right out of college has a lot more to do with your parents’ social position than anything you did in college.

Inconveniently Untrue Inconvenient Truths (Part 2)

Last week, Richard Vedder managed to completely infuriate me with 12 supposedly inconvenient truths about American higher education, most of which turned out to be, shall we say, exceptionally problematic. I tackled the first four last Friday (and must certainly apologize for making you wait, with baited breath I'm sure, for the next installment). Resuming where we left off . . . .

Inconvenient Truth #5 Undergraduate Students Are Often Neglected

There's no question that this is probably true at top-shelf research institutions. In grad school I saw excellent (and well-funded) researchers who couldn't teach with anything resembling competence sail through the tenure process. But saying that this is a problem with American higher education generally is like saying excessive speed is characteristic of the Italian auto industry. That Bugatti Veryon may break 260 miles per hour, even with James May behind the wheel. But a total of 300 were ever made. The run-of-the mill Fiat? Not so much. Admit it: that Alfa Romeo sitting in the garage isn't even running.

And that's where American higher education finds itself as well. Undergraduates may be low priority at Ph.D.-oriented institutions. But according to the Carnegie Foundation's classifications, those make up only 15% of primarily 4-year institutions. If the remaining institutions are giving undergraduates short shrift, who are they paying attention to? It surely isn't research: in 2000, despite making up more than three-fourths of the institutions that received federal research grants, non-research/doctoral institutions received only 15% of grant money.

Inconveniently Untrue Inconvenient Truths (Part 1)

Richard Vedder, in a blog post at the Chronicle of Higher Education, suggested "12 Inconvenient Truths About American Higher Education" that he plans on expanding in a series of essays. These claims, if true, are inconvenient for defenders of traditional higher education; Vedder suggests that these claims "suggest cumulatively American universities have a lot of problems." The problem is that they are mostly not true, or at least can't support the conclusions that Vedder reaches from them. Which is inconvenient for him and those who think the free market can do better with higher education than it did with mortgage lending.

Inconvenient Truth #1: College Costs Are Rising Both for Students and Society

This one we can't stop hearing about. Vedder maintains not only that tuition is up but also that "higher education absorbs more than triple the share of the nation’s productive efforts than it did when John F. Kennedy was president." I tackled the increasing cost concerns back in November, and have since updated the analysis (and made it fancy and interactive!). Basically, and assuming that my university is representative, costs haven't increased at state universities; tuition has gone up only because state funding has been slashed.

The increasing share of (I assume; Vedder doesn't really define "productive efforts") GDP, that reflects not a problem with universities but their success in the major goal of post-World War II educational policy: increasing access. The percentage of 18 to 34-year-olds attending college rose from 10.8% in 1963 to 23.8% in 2010. In 2010, 2% of those 35 and over were enrolled in college; in 1963 the Census Bureau didn't bother including them in the data. That alone accounts for most of the rising share of GDP. We allow more people to go to college today than we did in the golden era of the "American Century" and are somehow shocked—shocked!—to find that we spend more of our resources on education.