How did you become interested in social science?

We were talking in class yesterday about how each of us became interested in social science, specifically sociology. What kind of articles did we admire or remember? A few people mentioned some qualitative articles they had read and the rest of the class nodded along with the professor. Yes, qualitative articles are what we remember most, those quantitative articles are just a mess of numbers that never go far enough to analyze what is behind the numbers. I exaggerate, a bit.

I’m not surprised by this attitude; I’ve taken enough sociology classes at UTK to know that the slice of the department I’ve seen is very qualitative. There’s nothing wrong with this, most of my own research is qualitative. But there was a tiny taste of condescension that made me want to object. I can’t get it out of my mind or mouth.

But first some context. Most of the social sciences today are dominated by quantitative research ¹. There are a lot of reasons for this, some philosophically justified, some just dumb historical luck and habit. The argument, pro or con, for quantitative and qualitative research will have to be saved for another day. Let’s just conclude that a lot of qualitative researchers feel put upon to shape their research to the prevailing winds of disciplinarity which always seem to blow in a quantitative direction. I have some of the same war stories to share from my own personal experience. It’s disheartening and it makes people defensive, so when they find a safe place to complain they usually do.

But, for a moment, I want to defend quantitative research. First, there is the problem of argument by anecdote and personal experience. There is a value in finding the meaning and coming to an understanding of subjective experience, whether your own or another person’s; an argument that many philosophers and methodologists have made quite cogently. But sometimes the world is just too damn persistent and stable. Not everything boils down to an interpretation. The Earth orbits the sun, climate change is real, and evolution is a true theory. Sometimes the statistical or mathematical weight of experience is impossible to ignore. (I will bracket, for now, the question of whether the standards of the physical sciences should be applied to the human sciences.)

Another problem I have is the dreadful ignorance of statistics by both those who are enamored of qualitative methods and the general public. I’ve often thought that we would all be much better off as a country if high school students studied statistics instead of calculus. Calculus presents us with exact models of physical reality that work very dependably but the ‘real world’ of human interactions is stochastic, sometimes things just don’t go the way we modeled. The error terms never completely disappear (not that they do in physical sciences either, but I don’t remember talking about error terms when I was doing physics problem sets on inclined planes).

My final point against going all in for qualitative research is the inherent dangers of story-telling ². I will use a recent example from the world of science journalism. Jonah Lehrer is a science writer whose career went down in dramatic flames last summer when it was discovered that he had plagiarized his own work and falsified some quotes. I’m on the fence about copying your own work because copying runs a fine-line alongside revising and improving. Falsifying quotes, that gets you kicked out of college and every newspaper or magazine I know of.

I was reminded of Lehrer’s story and my own worries about qualitative research today. [Janet Stemwedel wrote a blog post] that quoted and pointed me to another blog post about the appeal of story-telling and the Lehrer incident. Christopher Chabris writes:

In my opinion, the fabrications and the scientific misunderstanding are actually closely related. The fabrications tended to follow a pattern of perfecting the stories and anecdotes that Lehrer – like almost all successful science writers nowadays – used to illustrate his arguments. Had he used only words Bob Dylan actually said, and only the true facts about Dylan’s 1960s songwriting travails, the story wouldn’t have been as smooth. It’s human nature to be more convinced by concrete stories than by abstract statistics and ideas, so the convincingness of Lehrer’s science writing came from the brilliance of his stories, characters, and quotes. Those are the elements that people process fluently and remember long after the details of experiments and analyses fade.

We expect, and respond, at a deep, psychological level to a good story. When we say something ‘makes sense’ we are saying that it tells us a good story about ourselves and the world. This is the danger I feel, creeping up behind me, whenever I start to think that qualitative research is where it’s at.

Dismissing quantitative research because we can’t get a handle on it or because the analysis doesn’t go far enough (more synonyms for telling us a good story) is foolish. We need more experience, more knowledge, even more truth (although it pains me, a tiny bit, to use that loaded word in this context) about the world. Numbers, statistics and quantitative methods are one way to test our hard won research against the world. They are not the only way to know or learn, but they are valuable.